The phrase "a difficult time building" is correct and usable in written English.
It can be used when describing challenges or struggles someone is facing in the process of creating or constructing something, whether it be physical, emotional, or conceptual.
Example: "She had a difficult time building her confidence after the setback."
Alternatives: "struggling to build" or "having trouble constructing".
King had a difficult time building around his star, Allen Iverson, and made a handful of questionable moves.
A Denver native who attended Colorado College, she earned master's and doctoral degrees in marine ecology from the University of Washington and Harvard and came to Oregon State in 1977, when it was a given that female scientists who wanted to have children would have a difficult time building successful research careers.
MVNOs typically have a difficult time building a customer base.
You will not be able to achieve the so-called "product market fit" early on, and you will have a difficult time building a long-lasting company.
This could be considered a blessing in disguise, as a storage-centric social network would have probably had a difficult time building a substantial userbase, and may have simply resulted in more lost time and money.
For these reasons, the U.S., despite its pronouncements to the contrary, is having a difficult time building a strong consensus around the Copenhagen Accord.
Indeed, Wal-Mart has had a surprisingly difficult time building up its own business.
Without that help, Netflix would have had a very difficult time building the breakthrough streaming media features that have differentiated them as quickly as they needed while also attracting some of the best talent in the valley.
Fitness clubs have a difficult time applying their muscle building, fast-paced fitness programs into this market.
Having committed $10 billion to building the A380, Airbus would have a difficult time developing a new jet to match the Sonic Cruiser.
When that happens, he said, shareholders typically have a difficult time obtaining financing for share loans and the building itself has trouble refinancing the underlying mortgage.
