Running Python from Java

 Running Python from Java on Linux

Problem statement

Java, a general purpose object-oriented language, is self-sufficient to provide most of the functionalities as available in any other programming language. But an application may require to use some feature which is managed better in a different language or product. An application designer may want to manage such features externally, for instance an external cache or a data base. One such important feature  is running analytical models. Data scientists use various analytical models written in R, H2O, pmml , python etc. In the world of machine learning, data science and analytics, it is needed that such application is capable of running the analytical models during runtime. In Java various implementations for these languages are available so that their native code makes use of the JVM libraries without the extensive data transfer, like rejnin for R. For python we have interpreters like Jython. Jython is basically python written in Java. The problem with interpreters like Jython is that either they are slow or they have limited support.

Solution

To run the python models in Java we prefer to use the OS native implementation of python and then calling the python scripts from within the java program as a system call. This enables us to use all the functionality available in python and we are not restricted by the releases. All the new updates or new versions can be made available once installed on the base OS. But for our data science friends who want to use some specific libraries like numpy, sklearn, SciPy, pandas etc. they can use the native distributions of python like Anaconda which enables them to use all such libraries. Since the OS under consideration is Linux, I’m going to explain how to achieve this on a Linux environment,

We can use one of the following two approaches to call the native python on Linux OS from java:

 

Runtime class method:

Every Java application has a single instance of class Runtime that allows the application to interface with the environment in which the application is running. This class provides methods to run processes on the native environment hence allowing us to use the native Python installed on the Linux environment. We are going to use 2 of these methods:

public static Runtime getRuntime() : returns the instance of Runtime class.

public Process exec(String command) : executes given command in a separate process.

Usually invoking a python script from Linux environment would look like this :

$ python myPythonScript.py

In this approach we will create the above command as a String in our Java program. Then after getting the runtime using getRuntime() method we can execute this command using the exec(command) method. Hence, executing the python script in the process. Let us assume that the script is expected to produce some result and we intend to use the same in our java program. For this we need to make sure that the result is printed in the python script as a last step and we can read the same from the output stream of the process using the getInputStream() method.

Our Java program looks like :

import java.io.*;
import java.util.Scanner;                                                       
class TestPythonRunTime {                                                      
public static void main(String a[]) {                                          
	try {                                                                   
		String returnValue = "";                                        
		// We are using python from anaconda                            
		String command = "/anaconda3/bin/python ";
		/* any parameters needed including the python script path can be 
                   added in the command as below */
		for (int i = 0; i < a.length; i++)
			command = command + " " + a[i];
		// executing the script
		Process p = Runtime.getRuntime().exec(command);
		Scanner stdin = new Scanner(new BufferedInputStream(p.getInputStream()));
		while (stdin.hasNext())
			returnValue = returnValue + stdin.nextLine();
		stdin.close();
		System.out.println("When called from Java " + returnValue);
	} catch (Exception e) {
		System.out.println(e);
	}
}
}

Our Simple Python script that prints a message looks like :

# This program prints a message
print ("Python Works !!")

After compiling the java class and running from command line the result looks like :

$java TestPythonRunTime /home/user/scripts/TestPython.py
When called from Java Python Works !!

 

ProcessBuilder class method:

This class is used to create operating system processes. This is the feature we can ride on to run a python script from our java program. Similar to runtime this class also provides a few methods to run processes on the native environment. The method we are using is:

java.lang.ProcessBuilder.start() : Starts a new process using the attributes of process builder

In this approach we will create the above command as an ArrayList< String > in our Java program. Then we create a new process for this command and start the process using start() method. Hence, executing the python script in the process. Let us use the similar python script as in the above case that is expected to produce some result and we intend to use the same in our java program. Again we need to make sure that the result is printed in the python script as a last step and we can read the same from the output stream of the process using the getInputStream() method. To catch any exception occurred during the script invocation can be caught as the standard error using the getErrorStream() method.

 

Our Java program looks like :

import java.io.*;
import java.util.Scanner;
import java.util.ArrayList;


class TestPythonPB {

public static void main(String a[]) {
	try {
		if(a.length <1 )
			return;
		String returnValue = "";
		ArrayList argsList = new ArrayList();


		// We are using python from anaconda
		argsList.add(0,"/anaconda3/bin/python");

		/* any parameters needed including the python script path can be added in the list as below */
		for( int i = 0 ;i < a.length;i++  )
	        argsList.add(i+1, a[i]);

		// executing the script
		ProcessBuilder pb = new ProcessBuilder( argsList);
		Process p = pb.start();
		
		// catching the standard error
		Scanner stderr = new Scanner(new BufferedInputStream(p.getErrorStream()));
		if (stderr.hasNext()) {
			System.out.println("Error while running python script.");
			while (stderr.hasNext()) {
				System.out.println(stderr.nextLine());
			}
			stderr.close();
			return;
		} else
			stderr.close();
		
		// reading the output
		Scanner stdin = new Scanner(new BufferedInputStream(p.getInputStream()));
		while (stdin.hasNext())
			returnValue = returnValue + stdin.nextLine();

		stdin.close();
		System.out.println("When called from Java using PB " + returnValue);

	} catch (Exception e) {
		System.out.println(e);
	}
}
}

Using the same Simple Python script :

# This program prints a message
print ("Python Works !!")

After compiling the java class and running from command line the result looks like :

$java TestPythonPB /home/user/scripts/TestPython.py
When called from Java using PB Python Works !!

 

This was a small attempt to show how simply we can run python scripts from our java programs. The approaches above definitely deviate java from being a platform independent programming language but with the limited support available for such kind of requirements it will always be a trade-off between feasibility and flexibility. It is entirely up to the application designers to decide on which approach to use as per their application needs.

Share This Post
Have your say!
4 0

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>