The Grocery Kingdom is mentioned by Finn in the episode "Five Short Graybles." He and Jake pretend to go there in order to see what BMO does when they are not at the Tree Fort. It is also mentioned in the episode "Varmints" by Marceline, as she describes it to have many free samples for her to suck the red out of. It is unknown if it is an actual kingdom.
The Grocery Kingdom is featured in Issue 20 and Issue 22 of the Adventure Time comic when Billy goes there to get ice cream for a party. It is depicted as a vast multi-level grocery store with escalators and ski lifts to get customers to various areas. However, it is unclear as to whether this is how the Grocery Kingdom actually looks.