The phrase "all that sturdy" is correct and usable in written English.
It can be used to describe something that is not very strong or durable, often in a somewhat informal context.
Example: "I thought the table would hold my weight, but it turns out it's not all that sturdy."
Alternatives: "not very strong" or "not particularly durable."
But vegetable samosas came wrapped in pastry that was all too sturdy; and like bar food, rings of squid were battered and fried, but with Indian spices.
This is peak season for squash, when the end of the zucchini overlaps with the beginning of the hard-shells, all those sturdy varieties that can carry a seasonally sensitive cook through the most barren months.
For all the sturdy runs from Gooch and Atherton, Smith and Lamb, and Azharuddin's memorable effort, David LLoyd was in no doubt that Tendulkar should be the first ever Boy of the Match winner.
All sought sturdy formats for painting that would face up to the masters while eschewing the he-man egomania that marred much of the "second generation" New York School.
He received all the sturdy, self-denying New England values, but also a measure of leeway.
For all the sturdy proclamations and occasional bluster in Mendelssohn's "Reformation" Symphony, the real drama occurs at moments when intense dynamism yields to blessed simplicity.
Implicitly, we all understand that knowledge is sturdier, more important, and more virtuous than beliefs or opinions or suspicions.
The Andante of the Beethoven had a buoyancy that the orchestra rarely achieved under the all too sturdy hand of Kurt Masur; the strings came to life with singing lines and plush textures.
There was evidence of this in the program I heard last Tuesday, which combined Beethoven's Ninth with a new work by John Adams, "On the Transmigration of Souls". The Andante of the Beethoven had a buoyancy that the orchestra rarely achieved under the all too sturdy hand of Kurt Masur; the strings came to life with singing lines and plush textures.
