We continue to have this problem. Database threads get locked up every single night and all sites go down. Our DBA notices connection will be very few then ramp up from, say, 2 to 245 right around the timeframe we get the ColdFusion hang. Also, he notices all these connections remain OPEN. What is going on? Also, is 245 open connections a lot? I've stress tested our system with hunderds of concurrent connections and I can never duplicate the issue.
I changed my database to uncheck "Maintain Connections" and still we get this problem. I have 9 datasources and I only made the change to one, the biggest one where the BLOCKED theads are happening, should I change them all? I just can't figure out where this coming from. Other things we've tried: changing the NIC driver, disabled IPv4 and TCP checksum offloading (Wireshark pointed us in this direction but still hasn't made any difference), logs really aren't giving us any data. Traffic on the site seems to be almost nothing sometimes when these events occur.
Other thoughts I've had: could the JDBC driver be the problem? We are running Oracle 11.2.0.1 (separate server, same network). My web boxes are all in VMware on a host with 40 other machines. IT dept says they cannot find anything running around the times of the database connection issue. I've checked the Windows 2008 "tasks" and can't find anything there. We have no ColdFusion tasks running.
Oddly, I've even seen this issue on my staging and devlopment machines. I've received a "slow server" alert when there weren't even any requests running!! Zero! Does this mean the skimmers and other native timers/whatever inside CF can't find a live connection? Just weird, weird, weird.
The problem is always at night, so of course that's a clue. But when I look through IIS raw logs nothing jumps out. If something was hitting my server, I would expect to see a ton of logs pointing to a source at those exact times, but there is nothing obvious. What else could be causing all the database connections to be used and held open if not traffic?