We study the response of the low-degree, solar p-mode frequencies to the unusually extended minimum of solar surface activity since 2007. A total of 4768 days of observations collected by the space-based, Sun-as-a-star helioseismic GOLF instrument are analyzed. A multi-step iterative maximum-likelihood fitting method is applied to subseries of 365 days and 91.25 days to extract the p-mode parameters. Temporal variations in the l=0, 1, and 2 p-mode frequencies are then obtained from April 1996 to May 2009. While the p-mode frequency shifts are closely correlated with solar surface activity proxies during the past solar cycles, the frequency shifts of the l=0 and l=2 modes increase from the second half of 2007, when no significant surface activity is observable. On the other hand, the l=1 modes follow the general decreasing trend of solar surface activity. The different behaviors between the l=0 and l=2 modes and the l=1 modes may be interpreted as different geometrical responses to the spatial distribution of the solar magnetic field beneath the surface of the Sun. The analysis of the low-degree, solar p-mode frequency shifts indicates that the solar activity cycle 24 started in late 2007, despite the absence of activity on the solar surface.