Here's some info: the coefficient of thermal expansion for brass is 19 parts/million per degree Celsius. For an example, let's calculate the effect of a 100 degree Fahrenheit temperature change on brass track. 100 degrees F is equivalent to 55.6 degrees C, so our formula would be 19 X 55.6 / 1000000 which equals .0010564 feet per foot of track. For 100 feet of track, the expansion would be 0.10564 feet, or about 1 1/4 inch.
 
Stainless steel is less of a problem as its coefficient of thermal expansion is just 17.3. A 100 degree F change on 100 feet of stainless steel track would be about 1 5/32 inch.
 
Aluminum is a little worse; it's coefficient is 23. A 100 degree F change on 100 feet of aluminum track would be a shade over 1 1/2 inch.
 
Again admitting I have no real outdoor experience, these numbers seem small to me. An inch of movement in 100 feet of track from deep frost to scorching sun? I would think that keeping a few rail joiners unclamped would allow for this kind of movement.  I ask someone who has had an actual problem with thermal expansion tell me how flawed my conclusion is!