問題描述:
程序中需要使用JDBC的方式同時訪問Hive和MySQL,發現在獲取MySQL連接的時候(DriverManager.getConnection方法)出現Hive的類中的Bad URL format異常。
代碼如下:
public Connection getMySqlConn() {
Connection conn = null;
try {
Class.forName("com.mysql.jdbc.Driver");
conn = DriverManager.getConnection(ConstantUtil.mysqlUrl, ConstantUtil.mysqlUsername, ConstantUtil.mysqlPassword);
} catch (ClassNotFoundException e) {
logger.error(e.getMessage(), e);
} catch (SQLException e) {
logger.error(e.getMessage(), e);
}
return conn;
}
異常如下:
java.lang.IllegalArgumentException: Bad URL format
at org.apache.hive.jdbc.Utils.parseURL(Utils.java:185)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:84)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:104)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at com.xxx.util.MySqlUtil.getMySqlConn(MySqlUtil.java:33)
at com.xxx.interceptor.TransformInterceptor.refreshApp(TransformInterceptor.java:151)
at com.xxx.interceptor.TransformInterceptor.initialize(TransformInterceptor.java:42)
at org.apache.flume.interceptor.InterceptorChain.initialize(InterceptorChain.java:74)
at org.apache.flume.channel.ChannelProcessor.initialize(ChannelProcessor.java:68)
at org.apache.flume.source.PollableSourceRunner.start(PollableSourceRunner.java:70)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
原因分析:
DriverManager.getConnection方法會使用url去嘗試訪問每個註冊的JDBCDriver的connect方法,而JDBC的規範中要求該方法要判斷url是否能夠處理,不能處理要返回null。
然而,hive-jdbc在0.11.0版本中(具體受影響的版本範圍未分析),調用其connect方法時,在url前綴非hive的情況下,是直接拋出IllegalArgumentException,而這個異常DriverManager並不會捕獲,導致程序直接報錯,不再嘗試正確的MySQL的Driver。
解決辦法:
(1)新版本的HiveJdbcDriver已修改這個問題。即先判斷url前綴是否可以處理(處理不了返回null),並且將parseURL的異常修改爲SQLException的子類(DriverManager會捕獲)。
(2)將調用的代碼改爲,先獲取你需要的Driver,再調用具體Driver的connect方法,這樣DriverManager就不會去嘗試調用每個Driver的connect方法了。代碼如下:
public Connection getMySqlConn() {
Connection conn = null;
try {
Class.forName("com.mysql.jdbc.Driver");
Driver driver = DriverManager.getDriver(ConstantUtil.mysqlUrl);
java.util.Properties info = new java.util.Properties();
info.put("user", ConstantUtil.mysqlUsername);
info.put("password", ConstantUtil.mysqlPassword);
conn = driver.connect(ConstantUtil.mysqlUrl, info);
} catch (ClassNotFoundException e) {
logger.error(e.getMessage(), e);
} catch (SQLException e) {
logger.error(e.getMessage(), e);
}
return conn;
}