Here are my local favs:

"Stratford at the Harbor" is a great brunch cafe/restaurant. It overlooks the beautiful, walkable harbor.

"The Rooftop Bar" at Mission Pacific Hotel is open to the public on the 6th floor overlooking the ocean. Amazing views and a small lunch.

"Pannikan's" is also a great relaxing cafe in a beautiful historic building.

"Craft Coast Beer and Tacos" is a great place for good beer and delicious tacos. Relax on their wonderful patio.

Something sweet:

"Handel's Ice Cream"

"Top Gun House's Pie"