最近在写一个Java工具,其中调用了各种SHELL命令,使用了Runtime.getRuntime().exec(command);这个方法。但是在我调用某个命令执行操作时,程序就卡在那了,但是其他操作却能够正常输出,经过了一番查找和摸索,终于明白了原来Java在执行命令时输出到某个Buffer里,这个Buffer是有容量限制的,如果满了一直没有人读取,就会一直等待,造成进程锁死的现象,知道这一点,我们就可以在执行的过程中新开启几个线程来不断地读取标准输出,以及错误输出:
final Process p = Runtime.getRuntime().exec(command); new Thread(new Runnable() { @Override public void run() { while (true){ try { p.exitValue(); break; } catch (Exception e){ showInfo(System.err,p.getErrorStream()); } } } }).start(); new Thread(new Runnable() { @Override public void run() { while (true){ try { p.exitValue(); break; } catch (Exception e){ showInfo(System.out,p.getInputStream()); } } } }).start(); int exitValue = p.waitFor();
需要注意的是,在waitFor执行完成之前,错误输出和标准要分开处理。
Apache commons-exec提供一些常用的方法用来执行外部进程,Apache commons exec库提供了监视狗Watchdog来设监视进程的执行超时,同时也还实现了同步和异步功能,Apache commonsexec涉及到多线程,比如新启动一个进程,Java中需要再开三个线程来处理进程的三个数据流,分别是标准输入,标准输出和错误输出。
需要使用该功能需要引入commons-exec-1.3.jar包,目前最新的版本为1.3版本。
Apache commons-exec的官方网站:http://commons.apache.org/proper/commons-exec/
其中就有相应的示例Example,来详解如何使用该工具来执行shell命令,我们执行命令的相关代码:
final Long executeSubTaskId = subTaskExecuteContext.getSubTaskExecuteId(); final Long taskStatusId = subTaskExecuteContext.getTaskExecuteContext().getTaskExecuteId(); ByteArrayOutputStream outputStream = new MzByteArrayOutputStream(executeSubTaskId, taskStatusNotifyCenter, true); ByteArrayOutputStream errorStream = new MzByteArrayOutputStream(executeSubTaskId, taskStatusNotifyCenter, false); PumpStreamHandler streamHandler = new PumpStreamHandler(outputStream, errorStream); taskThreadPoolExecutor.setStreamHandler(streamHandler); CommandLine commandLine = new CommandLine(new File(executeShellPath)); final TaskProcessInfo taskProcessInfo = new TaskProcessInfo(subTaskExecuteContext.getTaskExecuteContext().getTaskId(), taskStatusId, executeSubTaskId); ProcessManagerDestroyer processManagerDestroyer = new ProcessManagerDestroyer( taskProcessInfo, processInfoManager); taskThreadPoolExecutor.setProcessDestroyer(processManagerDestroyer); try { taskThreadPoolExecutor.execute(commandLine, new DefaultExecuteResultHandler() { @Override public void onProcessComplete(int exitValue) { super.onProcessComplete(exitValue); LOG.info(String.format("Task Process info: %s succeed!", taskProcessInfo)); taskStatusNotifyCenter.notifyEventChanged(new TaskStatusEvent(TaskStatusEventType.TASK_FINISHED, new TaskEventObject(subTaskExecuteContext.getTaskExecuteContext().getTaskId(), subTaskExecuteContext.getTaskExecuteContext().getTaskExecuteId(), subTaskExecuteContext.getSubTaskExecuteId()))); } @Override public void onProcessFailed(ExecuteException e) { super.onProcessFailed(e); LOG.error(e); LOG.error(String.format("Task Process info: %s failed!", taskProcessInfo)); taskStatusNotifyCenter.notifyEventChanged(new TaskStatusEvent(TaskStatusEventType.TASK_FAILED, new TaskEventObject(subTaskExecuteContext.getTaskExecuteContext().getTaskId(), subTaskExecuteContext.getTaskExecuteContext().getTaskExecuteId(), subTaskExecuteContext.getSubTaskExecuteId()))); } }); } catch (IOException e) { throw new BusinessException(e); }
新建执行Process需要new两个ByteArrayOutputStream,一个用来记录标准输出流,一个用来记录错误输出流。为了及时清理ByteArrayOutputStream中的内容,可以选择性地将该输出流重写:
@Override public synchronized void write(byte[] b, int off, int len) { super.write(b, off, len); writeTimes++; writeLength += len; if (writeLength >= MAX_WRITE_LENGTH || writeTimes >= MAX_WRITE_TIMES) { updateStatus(); this.buf = new byte[32]; writeLength = 0; writeTimes = 0; } } @Override public void flush() throws IOException { super.flush(); updateStatus(); }
建立的ProcessManagerDestroyer用来任务创建或任务完成时,对任务的当前记录状态。
public class ProcessManagerDestroyer implements ProcessDestroyer { private final ProcessInfoManager processInfoManager; private final TaskProcessInfo taskProcessInfo; public ProcessManagerDestroyer(TaskProcessInfo taskProcessInfo, ProcessInfoManager processInfoManager) { this.taskProcessInfo = taskProcessInfo; this.processInfoManager = processInfoManager; } @Override public boolean add(Process process) { processInfoManager.addProcess(taskProcessInfo, process); return true; } @Override public boolean remove(Process process) { processInfoManager.removeProcess(taskProcessInfo); return true; } @Override public int size() { return processInfoManager.taskCount(); }
在Destroyer中新建的Process可以保存,并在以后调用destroy方法将其kill掉:
process.destroy();
最后建立的DefaultExecuteResultHandler监听器用来在任务执行完成或出现错误时,提示对应的信息,并发送事件。
执行shell时遇到的问题,初步看来,没有执行的权限?
org.apache.commons.exec.ExecuteException: Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "/Users/mazhiqiang/Downloads/1.sh" (in directory "."): error=13, Permission denied) at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:205) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Cannot run program "/Users/xxx/Downloads/1.sh" (in directory "."): error=13, Permission denied at java.lang.ProcessBuilder.start(ProcessBuilder.java:1042) at java.lang.Runtime.exec(Runtime.java:620) at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:61) at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:279) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:336) at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48) at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200) ... 1 more Caused by: java.io.IOException: error=13, Permission denied at java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:185) at java.lang.ProcessImpl.start(ProcessImpl.java:134) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1023) ... 7 more
最终发现生成的shell文件没有加上可执行文件的executable属性,以及shell命令文件头:
file.executable() !/bin/bash
应用过程中后面还有很多坑,等着我们去填......
相关推荐
hive 开发UDF 使用maven工程 引发jar包缺失 hive 开发UDF 使用maven工程 引发jar包缺失
包括以下jar包: Attributes Beanutils Betwixt Chain CLI Codec Collections Configuration Daemon DBCP DbUtils Digester Discovery EL Email Exec FileUpload IO JCI Jelly JEXL JXPath Lang Launcher Logging ...
hive JDBC jar包全家桶。由于项目使用,此jar包从国外下载费了好大劲,现分享给大家。 cdh6.3.2版本的
mr 程序依赖程序包 groupID org.apache.hadoop hadoop-common hadoop-hdfs hadoop-mapreduce-client-core hive 编程 hiveUDF 程序依赖程序包 groupID org.apache.hive hive-exec hive-common 同时需要hadoop的hadoop...
wexitstatus,pcntl_wtermsig,pcntl_wstopsig,pcntl_get_last_error,pcntl_strerror,pcntl_sigprocmask,pcntl_sigwaitinfo,pcntl_sigtimedwait,pcntl_exec,pcntl_getpriority,pcntl_setpriority,imap_open,apache_...
Navigating the Dangerous Waters of exec(),system(), and Backticks 27 Using System Binaries with the SUID Bit and sudo 28 Using System Resources 29 Usingescapeshellcmd() and ...
所有变量都是局部变量,为了使得定义的函数中可以使用外部变量,使用global语句。而你要将该变量的作用范围限制在该函数之内,使用static语句。 $g_var = 1 ; // 全局范围 function test() { global $g_var; // 这样...
Class.forName("org.apache.commons.collections.ArrayStack"); supportCommonCollection = true; } catch (ClassNotFoundException ex) { } try { Class.forName("org.apache.commons.digester.Digester"); ...
[INFO] ———————————————————————— [INFO] Reactor Summary for demo 0.0.1-SNAPSHOT: [INFO] [INFO] demo ……………………………………….. FAILURE [18:18 min] [INFO] admin-service ...
Configuring Some Common Web Servers O'Reilly's WebSite H NCSA httpd H Apache H G Examining Some Other File- and Site-Administration Issues G From Here G Chapter 11 Database Interaction ...
Here are some more specific instructions for common cases: NetBeans (Java SE 7 or newer): In the Projects window, right-click the Libraries node of your project, and select "Add JAR/Folder...". ...
# $(call find-copy-subdir-files,*,device/softwinner/polaris-common/rild/usb_modeswitch.d,system/etc/usb_modeswitch.d) PRODUCT_PROPERTY_OVERRIDES += \ ro.sw.embeded.telephony = false PRODUCT_...