Most race days in Brisbane are either on a Saturday or a Sunday, mostly Saturday though I think.
Take a look at these sites for the latest race events.
http://www.doomben.com/http://www.qtc.org/racing/calendar.cfm
It's a good day out even if you aren't a serious gambler. Lots of colour and movement, a few drinks and so long as the weather is good it's not a bad day out for a spot of fresh air without too much excertion

You can even take your kids if you must.