It is hard to recommend a starting time for someone else because it depends on the temperatures the day you hike, your pace, and how high a temperature you are willing to tolerate.
We went last Saturday (May 19), starting at 4:40 AM. It was about 70 at the start and temperatures were between 65 and 75 all the way. Our high temperature came around sunrise, about an hour into the hike. We were at least a couple of thousand feet up by then and we hit Long Valley at 9:10. Also, it was not a terribly hot day -- under 100 when we got back to the car, which is mild for Palm Springs.
We have done C2C in late June and encountered temperatures as high as the mid 90s on the way to Long Valley (and 115 back at the car). That is not hot enough to make me start before 4:30 AM, but we do manage a pace that keeps us rising fast enough to beat the heat.
I'm sure that I will be willing to consider earlier starts in a few years because it is inevitable that I won't be moving as quickly as I can move today. I liky Cy's approach of moving start times earlier for that obvious reason. The other approach would be to not go. I certainly don't want that to happen any time soon!
