Three seconds / five seconds / thirty ...
Goldfish have excellent memory. That's why they are frequently used in scientific studies of memory and learning.
On the first day, the goldfish took up to 20 minutes to learn to avoid the high-voltage end. But, put back again a day later, it took them only two minutes to give it a wide berth.
Fiona MacRae describes such an experiment (2006)
The fish, previously believed to have a memory of just a few seconds, can distinguish between different times of day and can also be taught to follow a routine, according to research.
Mark Henderson (2003)
When, if ever, science believed in "goldfish memory" is unclear, but it certainly didn't in 1966:
In one experiment inspired by that theory [that "memory" is formed in proteins], University of Michigan Biochemist Bernard Agranoff taught goldfish to swim over a barrier, then injected them with puromycin, an antibiotic that prevents protein synthesis. When the injection was given hours after learning, it had no effect, suggesting that memory proteins had already formed. Injected just before or just after training, the drug prevented learning.
Time, April 19, 1971
Heavier stuff still - two japanese make a serious attempt to figure out how well goldfish can remember where they swim:
We found that goldfish exhibited spatial learning in a maze, the hole board task, which resembles the Morris water maze.*
The present experiments provide a reasonable method to compare the spatial learning of fish with other animals. One of the characteristics of our maze task is that fish learned the spatial task quickly. They showed improvement within three training sessions. Such fast learning is similar to that of rats in the Morris maze. (...)
The present study showed that goldfish, like rodents, learned not only the fixed-route task, as in the Y or radial maze, but also the free-orienting (open field) task. This suggests that the capacity for spatial learning in goldfish is comparable to that of rodents.
Saito & Watanabe (2005)
* A "Morris water maze" is not really a maze but a little pool with opaque water. In this water you put a rodent. Somewhere in the pool is a small platform, just below the surface. In order to reach the safety of the platform, the rodent (who can't see it in the milky water) has to navigate using visual cues placed around the pool. By repeating this experiment one can quickly see how (or if) the animal learns the location of the platform. It is this kind of spatial learning that, according to Saito & Watanabe, goldfish are as good at as rodents.