Around 800 years before the series, in "Bonnibel Bubblegum", Gumbald, along with Lolly and Chicle, were created by a lonely young Bonnibel Bubblegum. Gumbald talked to his niece about building a candy city, but Bonnie, not liking the idea, made a butterscotch lake in the location where Gumbald intended to build the city. After sending Bonnie away, Gumbald along with Bonnibel's other intelligent relative-creations, Lolly and Chicle, were discussing a plan to overthrow her and build a candy city. He formulated a "happy juice" that would make Bubblegum more docile and simple-minded. Gumbald tricked Lolly and Chicle into eating happy juice-induced cupcakes turning them into Manfried and Crunchy, forgetting their previous selves. Bonnibel caught this. Gumbald voiced that Bonnie tried to keep him in line but stated he was her equal as she denounced him as evil. Gumbald tried to make Bonnie have the juice, but with the help of a peashooter, she shot the beaker with the happy juice while Gumbald was holding it. The happy-juice splashed onto Gumbald's face and turned him into the Punch Bowl and forgetting his previous self.
After Lumpy Space Princess's anti-elemental "lumps", reverted Ooo back to normal from the elemental spell. The Punch Bowl, Manfried, and Crunchy, were reverted back to their previous selves. The trio immediately resumed their plans from 800 years ago, getting revenge on their niece, and building the city.
The fully-constructed city first appears in "Gumbaldia", in which the name is revealed to be Gumbaldia. It is based around a ziggurat temple (the one that Finn and Fern previously visited in "Three Buckets") which is now a fully renovated palace made modeled after a cake for Gumbald, Lolly, and Chicle. The city is constructed entirely from candy and cake and shares many similarities with the Candy Kingdom. Princess Bubblegum was planning on declaring war on Gumbaldia, but Finn and Jake went to confront Gumbald, with hopes of calling off the war. After seemingly making peace, Gumbald, Lolly, and Chicle splash Finn and Jake with some sort of liquid, but were told to give Bubblegum "a big hug." Princess Bubblegum was thrilled with a peace treaty, but disliked the smell of Finn and Jake. Peppermint Butler wiped off the liquid, and was transformed into a docile, baby-like peppermint. After this, Princess Bubblegum realized that Finn and Jake were splashed with Gumbald's Dum Dum Juice, with hopes of transforming her in a decapitation strike. Following this, Princess Bubblegum once again declared war on Gumbaldia.
Despite Gumbaldia showing its preparedness for war, there seem to be no practical military installations seen at all, such as forts or garrisons. The only things that actually fit in this category is the renovated temple and some slightly reinforced buildings. There also seem to be no fortifications surrounding the city as well, such as trenches, pillboxes, or gun batteries. The only things that actually seem to protect the city from an invasion are a mountain range, a sizable army of cake soldiers, and a handful of planes, armored vehicles, and SAMs (surface-to-air missiles).
In "Come Along With Me", both Gumbald and Princess Bubblegum were ready for war to fight for their respective kingdom. On the battle field, both sides have sizable armies with a number of allies each. After Princess Bubblegum sounds the war horn, Finn tries to convince her to give Gumbald one last chance to surrender before the fighting starts. She agrees after being reminded of her old friend, Shoko. As a last ditch attempt to stop the war, Jake unleashes the nightmare juice given to him and Finn by Nightmare Princess, forcing him, Finn, Princess Bubblegum, Gumbald, and Fern into a hallucinatory slumber.
Inside the dream realm, Princess Bubblegum and Gumbald continue to be hostile towards each other, but eventually Bonnie apologizes to Gumbald for not respecting his vision for the Candy Kingdom. Gumbald fake-accepts her apology and agrees to end the conflict, intending to betray her, but is tripped by Lolly and once again doused in his own happy juice solution, turning back into Punch Bowl. Lolly accepts the peace agreement in his place, and Princess Bubblegum proclaims that the two kingdoms shall live in harmony, leaving the war unfought. However, they are soon all interrupted by the sudden arrival of GOLB. Lolly led Gumbaldia in the fight against GOLB and told the army to set aside all past differences with the Candy Kingdom.