When Google announced that it was officially including page speed as a ranking factor in determining results, I wondered how they determined page speed. Despite my Google searching prowess, I was unable to find a definitive answer to this question. A few weeks ago however, I was working on a client site and testing invalid urls; urls that weren't indexed by Google or accessed by the general public. The client reported that his Google page speed statistics in Google webmasters improved from 3.5s to 1.5s. Interestingly, the urls displayed as page speed examples were the urls I was accessing and repeatedly refreshing for testing purposes. The only way Google could have been determining those page speeds was through the Google Toolbar. After some further research, I can across this announcement from Google back in 2009:

"The load time data is derived from aggregated information sent by users of your site who have installed the Google Toolbar and opted-in to its enhanced features."

So there you have it. Google page speed is determined by the site load times of users who are using the toolbar with enhanced features. If you have a fast connection, be sure to use the Google toolbar when accessing your own site!