Why does the average temperature start dropping about a month after the daylight length starts dropping?

In the northern hemisphere, the longest day is on June 21. In temperate latitudes, the hottest week is often in the last half of July, when you average from year to year. Why does the temperature increase/decrease lag the day length increase/decrease in this way?