大家好，我是帥氣小伙。今天為大家分享一下有關kettle需要導入第三方jar包時，動態加載的經驗。我們在使用kettle的時候，經常需要用一些第三方的jar包，例如 mssql的驅動，我們平?？梢园阉诺絢ettle_home的lib目錄下，然后重啟。這樣其實挺不方便的，尤其是對于集成了kettle調度的應用來說。又是往往就是一個jar包的問題，導致平臺要重啟N次，顯示會對一些生成環境的業務造成巨大的影響。于是實現kettle運行時jar包的動態加載還是有價值的。
思路
- 為什么kettle客戶端重啟就能加載到lib的jar包呢？
- 1.kettle的入口 spoon.bat/spoon.sh
if %STARTTITLE%!==! SET STARTTITLE="Spoon"
REM Eventually call java instead of javaw and do not run in a separate window
if not "%SPOON_CONSOLE%"=="1" set SPOON_START_OPTION=start %STARTTITLE%
@echo on
%SPOON_START_OPTION% "%_PENTAHO_JAVA%" %OPT% -jar launcher\pentaho-application-launcher-5.4.0.1-130.jar -lib ..\%LIBSPATH% %_cmdline%
@echo off
if "%SPOON_PAUSE%"=="1" pause
顯然，kettle的入口程序是 pentaho-application-launcher-5.4.0.1-130.jar，然后傳入了lib參數
- 2.反編譯 pentaho-application-launcher-5.4.0.1-130.jar
public static void main(String[] args) throws Exception {
Parameters parameters = Parameters.fromArgs(args, System.err);
URL location = Launcher.class.getProtectionDomain().getCodeSource().getLocation();
File appDir = FileUtil.computeApplicationDir(location, new File("."), System.err);
File configurationFile = new File(appDir, "launcher.properties");
Properties configProperties = new Properties();
try {
configProperties.load(new FileReader(configurationFile));
} catch (Exception var13) {
;
}
Configuration configuration = Configuration.create(configProperties, appDir, parameters);
if (configuration.isUninstallSecurityManager()) {
System.setSecurityManager((SecurityManager)null);
}
Iterator i$ = configuration.getSystemProperties().entrySet().iterator();
while(i$.hasNext()) {
Entry<String, String> systemProperty = (Entry)i$.next();
System.setProperty((String)systemProperty.getKey(), (String)systemProperty.getValue());
}
List<URL> jars = FileUtil.populateClasspath(configuration.getClasspath(), appDir, System.err);
jars.addAll(FileUtil.populateLibraries(configuration.getLibraries(), appDir, System.err));
URL[] classpathEntries = (URL[])((URL[])jars.toArray(new URL[jars.size()]));
ClassLoader cl = new URLClassLoader(classpathEntries);
Thread.currentThread().setContextClassLoader(cl);
if (StringUtil.isEmpty(configuration.getMainClass())) {
System.err.println("Invalid main-class entry, cannot proceed.");
System.err.println("Application Directory: " + appDir);
System.exit(1);
}
if (configuration.isDebug()) {
System.out.println("Application Directory: " + appDir);
for(int i = 0; i < classpathEntries.length; ++i) {
URL url = classpathEntries[i];
System.out.println("ClassPath[" + i + "] = " + url);
}
}
Class<?> mainClass = cl.loadClass(configuration.getMainClass());
String[] newArgs = new String[args.length - parameters.getParsedArgs()];
System.arraycopy(args, parameters.getParsedArgs(), newArgs, 0, newArgs.length);
Method method = mainClass.getMethod("main", String[].class);
method.invoke((Object)null, newArgs);
}
- 3.關鍵性代碼
List<URL> jars = FileUtil.populateClasspath(configuration.getClasspath(), appDir, System.err);
jars.addAll(FileUtil.populateLibraries(configuration.getLibraries(), appDir, System.err));
URL[] classpathEntries = (URL[])((URL[])jars.toArray(new URL[jars.size()]));
ClassLoader cl = new URLClassLoader(classpathEntries);
Thread.currentThread().setContextClassLoader(cl);
也就是說，只需把jar包動態加載到Thread.currentThread()的classpath就可以了。
實現方式
private static ConcurrentHashMap jarMaps = new ConcurrentHashMap();//防止重復加載
/**
* 動態加載jar到kettle運行時classpath
* @param libs
* @return
*/
public static List<String> loadExtLib(List<String> libs){
List<String> success = new ArrayList<String>();
for(String path : libs){
ClassLoader cl = Thread.currentThread().getContextClassLoader();
try {
//反射調用 addURL
Method addURL = URLClassLoader.class.getDeclaredMethod("addURL", new Class[] {URL.class});
addURL.setAccessible(true);
File file = new File(path);
List<String> paths = new ArrayList<String>();
findJars(file,paths);//尋找路徑下的jar包
for(String jarPath : paths){
File jar = new File(jarPath);
FileInputStream inputStream = new FileInputStream(jar);
String jarHash = md5HashCode(inputStream);
if(!jarMaps.containsKey(jarHash)){
jarMaps.put(jarHash,jarPath);
URL url = jar.toURI().toURL();
addURL.invoke(cl, new Object[] { url });
success.add(jar.getName());
log.info("成功加載："+jarPath);
}
inputStream.close();
}
} catch (Exception e) {
e.printStackTrace();
log.error("加載kettle類庫失敗,原因",path,e);
}
}
return success;
}
/**
* 遍歷文件jar包
* @param file 文件夾
* @param jarPaths jar包路徑集合
*/
public static void findJars(File file,List<String> jarPaths){
File[] files = file.listFiles();
for(File a :files){
String path = a.getPath();
if(a.isFile() && (path.endsWith(".jar") || path.endsWith(".zip"))){
jarPaths.add(path);
}
if(a.isDirectory()){
findJars(a,jarPaths);
}
}
}
/**
* java獲取文件的md5值
* @param fis 輸入流
* @return
*/
public static String md5HashCode(InputStream fis) {
try {
//拿到一個MD5轉換器,如果想使用SHA-1或SHA-256，則傳入SHA-1,SHA-256
MessageDigest md = MessageDigest.getInstance("MD5");
//分多次將一個文件讀入，對于大型文件而言，比較推薦這種方式，占用內存比較少。
byte[] buffer = new byte[1024];
int length = -1;
while ((length = fis.read(buffer, 0, 1024)) != -1) {
md.update(buffer, 0, length);
}
fis.close();
//轉換并返回包含16個元素字節數組,返回數值范圍為-128到127
byte[] md5Bytes = md.digest();
BigInteger bigInt = new BigInteger(1, md5Bytes);//1代表絕對值
return bigInt.toString(16);//轉換為16進制
} catch (Exception e) {
e.printStackTrace();
return "";
}
}
寫個controller調用下就ok了。
@RequestMapping("/load/lib")
@ResponseBody
public List<String> loadLib(){
List<String> success = KettleUtils.loadExtLib(config.getLibs());
return success;
}
總結
就說這么多，有關于kettle調度這個項目，我盡量把它完善下吧，等一個spring-boot-starter-kettle吧。
