I had an odd incident happen a week or so ago involving a few cPanel servers.

I was attempting to standardize our configuration, and for some reason mysql started failing. The error message

070309 11:49:10 /usr/sbin/mysqld: Can't open file: 'host.MYI'. (errno: 142)
070309 11:49:10 Fatal error: Can't open privilege tables: File '/usr/share/mysql/charsets/?.conf' not found (Errcode: 2)
070309 11:49:10 Aborting
070309 11:49:10  InnoDB: Starting shutdown…
070309 11:49:12  InnoDB: Shutdown completed
070309 11:49:12 /usr/sbin/mysqld: Shutdown Complete
070309 11:49:12  mysqld ended

Yeah, not so good. I tried a few troubleshooting steps, and the error seemed to be related to finding 'charset #83', which according to my research was utf-8. That's very common, why the heck would it have a problem with that?

I was able to hack it up a little bit, but eventually I found the real solution

SOLUTION:

The mysql version was incorrect on the server. I had accidentally DOWNGRADED from mysql 4.1 to mysql 4 without realizing it. Here's how it happened:

  1. Mysql 4.1 was installed on the server
  2. In an attempt to standardize the server configuration, I copied /var/cpanel/cpanel.config from another host
  3. This 'new' cpanel.config contained the line:
    mysql-version=4
  4. When I next ran /scripts/upcp, it updated the mysql version to version 4.0.x.
  5. Once the mysql daemon was restarted, it could no longer read its own privilege tables, because the charset was specific to mysql 4.1
  6. The fix was to change the offending line in /var/cpanel/cpanel.config to
    mysql-version=4.1
    then run /scripts/mysqlup

Voila! That's a few hours of my life I won't get back any time soon.

Moral of the story: Config files don't always play nice..!