Difference between revisions of "SoBigData: step-by-step procedure for algorithm integration"
From D4Science Wiki
(→SoBigData: step-by-step procedure for algorithm integration) |
|||
| Line 23: | Line 23: | ||
# Run Test_App. | # Run Test_App. | ||
| − | In the template there is an example of how to call a python script during the process procedure. | + | In the [[#App.java | template]] there is an example of how to call a python script during the process procedure. |
== App.java == | == App.java == | ||
| Line 54: | Line 54: | ||
@Override | @Override | ||
| − | public void init() throws | + | public void init() throws Excepa]tion { |
//IF YOU NEED TO DO SOMETHING BEFORE THE EXECUTION. | //IF YOU NEED TO DO SOMETHING BEFORE THE EXECUTION. | ||
Revision as of 15:55, 26 May 2016
This very detailed step-by-step procedure was produced by R. Trasarti (KDD):
- Install Eclipse
- Open Eclipse, Skip to 4 if Maven is already installed (last versions of Eclipse have it)
- Copy the address for maven installation: http://download.eclipse.org/technology/m2e/releases
- In Eclipse, install the new plugin (help->install new Software). It takes minutes to load the dependencies.
- Create a new project: File->New->Other->Maven->Maven Project
- Select the default project type ([…]Quickstart 1.1)
- Specify as GroupID = “app”, ArtifactID = “template” (important: Lower Camel Case)
- Open pom.xml, click on pom.xml tab (below)
- go to: https://wiki.gcube-system.org/gcube/How-to_Implement_Algorithms_for_the_Statistical_Manager
- copy the dependencies in the first box under Maven coordinates
- paste them into the pom.xml (under dependecies after the existing ones)
- go to: https://wiki.gcube-system.org/gcube/Talk:Creating_gCube_Maven_components:_How-To
- copy all the setting box
- go to your home and move to “.m2/“ folder than create the file settings.xml and paste the text in it. The system will download the dependencies.
- download http://goo.gl/aWlPV5
- Extract and copy the inner cfg folder (there are two with the same name), and paste the folder in the project
- delete the content of the cfg/dynamictransducerers.properties file. save and close it.
- Implement your algorithm following the template (see attached files)
- Open cfg folder in the project and add in transducerers.properties add “APP=app.template.App”
- Run Test_App.
In the template there is an example of how to call a python script during the process procedure.
App.java
package app.template; import java.io.BufferedReader; import java.io.InputStreamReader; import org.gcube.dataanalysis.ecoengine.datatypes.PrimitiveType; import org.gcube.dataanalysis.ecoengine.datatypes.StatisticalType; import org.gcube.dataanalysis.ecoengine.datatypes.enumtypes.PrimitiveTypes; import org.gcube.dataanalysis.ecoengine.interfaces.StandardLocalExternalAlgorithm; /** * Hello world! * */ public class App extends StandardLocalExternalAlgorithm{ public static final String test_string_name="test_string"; public static final String resultKey = "result"; private String result = null; @Override public String getDescription() { //RETURN A STRING WITH THE DESCRIPTION OF YOUR ALGORITHM return null; } @Override public void init() throws Excepa]tion { //IF YOU NEED TO DO SOMETHING BEFORE THE EXECUTION. } @Override protected void process() throws Exception { //READ THE INPUTS String test_string = getInputParameter(test_string_name); System.out.println("Input: "+test_string); //YOUR ALGORITHM MUST BE HERE! result = test_string+" (OK)"; //<== START - EXAMPLE OF HOW TO CALL A PHYTON SCRIPT ==> // String pythonCmd = getInputParameter("python_location"); // ProcessBuilder pb = new ProcessBuilder(pythonCmd, "param1", "param2", ...); // Process pr = pb.start(); // // BufferedReader stdOutput = new BufferedReader(new InputStreamReader(pr.getInputStream())); // BufferedReader stdError = new BufferedReader(new InputStreamReader(pr.getErrorStream())); // String stdReader = null; // while((stdReader = stdOutput.readLine())!=null) { // System.out.println(stdReader); // } // while((stdReader = stdError.readLine())!=null) { // System.out.println(stdReader); // } //<== END - EXAMPLE OF HOW TO CALL A PHYTON SCRIPT ==> } @Override protected void setInputParameters() { //YOU MUST DEFINE YOUR INPUTS addStringInput(test_string_name, "Description", "DefaultInput"); } @Override public void shutdown() { //SOME OPERATION AT THE END OF THE PROCESS } @Override public StatisticalType getOutput() { PrimitiveType val = new PrimitiveType(String.class.getName(), result , PrimitiveTypes.STRING, resultKey, "This is the result of my algorithm"); System.out.println("Output: "+result); return val; } }
Test_App.java
package app.template; import java.util.List; import org.gcube.dataanalysis.ecoengine.configuration.AlgorithmConfiguration; import org.gcube.dataanalysis.ecoengine.interfaces.ComputationalAgent; import org.gcube.dataanalysis.ecoengine.processing.factories.TransducerersFactory; import org.gcube.dataanalysis.ecoengine.test.regression.Regressor; public class Test_App { private static AlgorithmConfiguration testConfigLocal() { AlgorithmConfiguration config = Regressor.getConfig(); config.setAgent("APP"); config.setParam(App.test_string_name, "HELLO WORLD?"); //config.setParam("python_location", "/Library/Frameworks/Python.framework/Versions/2.7/bin/python";); return config; } public static void main(String[] args) throws Exception { List<ComputationalAgent> trans = null; trans = TransducerersFactory.getTransducerers(testConfigLocal()); trans.get(0).init(); Regressor.process(trans.get(0)); trans.get(0).getOutput(); trans = null; } }
