To take a look in person, first find a female tree, since the males (which are much more widely planted in New York, for reasons soon to be clear) don't bear fruit. Botanically the golden ovoid isn't a fruit, but since the pulp is inedible, let's move on: Slice or squeeze it open and discard the pulp, retaining the pistachio-shaped structure inside. Use a nutcracker, gently, to remove the shell, which splits easily enough but is too brittle to be pried open expeditiously. The resulting "candy corn" can be sauteed in olive oil; loud pops will signal the separation of those brown half-skins, exposing bright green nuggets that want only a touch of salt. The texture is rather soft, like a less-flavorful chestnut.
There's also this: Ginkgo fruits stink. Nowadays male trees are more widely planted because they don't drop malodorous yellow globes that rupture on walkways or lurk in the soft grass. The smell is no challenge to the reign of the king of fruits, but then, for an extra buck my favorite durian vendor will take on the most labor-intensive part of the process. Ginkgo nut vendors likewise: The gentleman and lady in the photos at bottom are selling shelled nuts in Manhattan's Chinatown.
Ginkgo nuts
My harvest: From a tree on the Upper East Side
Vendors: Mott St. near the northeast corner with Hester St., Manhattan, autumn