I used to work in Potrero Hill and tested out some of these fab finds without little ones. Like the cupcakes and brownies at Baked: I had to taste them one by one over the course of many weeks. And the huge park? I had to play dogeball with other nerdy adults! You're welcome.
I suggest that you drive on over to Potrero Hill with another parent friend and a pair of toddlers and explore the kid-friendly sites. They are all within walking distance (if you like to schlep up and down big steep hills):
Urban Recess (Indoor Playspace)
I took a test-run of the awesome indoor playspace, Urban Recess, with my two-year old. He loved the wide open spaces, cozy book nooks, super fast slide (caution!), puzzles, ride-along toys and just about everything else. I liked the casual kitchen and friendly vibe. Located at 470 Carolina Street at 18th St. Closed Wednesday and weekend afternoons. Call for hours and programs or see their website: 415.701.7529
Public Park including Jackson Playground
This enormous sunny public park has fields for baseball and general running around, indoor and outdoor basketball courts and a totlot including sand area. Located at 17th Street (between Carolina and Arkansas streets)
Whole Foods (easiest lunch stop)
There is a full-service cafe on the underbelly of this great Whole Foods. Located at 450 Rhode Island Street.
Reward yourself for a fun morning of playtime (and a healthy lunch) by trekking up the hill. Cupcakes are divine and brownies are so rich (and large!) they could comfortably serve a family of four. Located at 1415 18th St up the hill (between Connecticut St & Missouri St).