Difference between revisions of "SoBigData: step-by-step procedure for algorithm integration"

From D4Science Wiki
Jump to: navigation, search
(→‎SoBigData: step-by-step procedure for algorithm integration)
 
(3 intermediate revisions by the same user not shown)
Line 2: Line 2:
 
This very detailed step-by-step procedure was produced by [http://www-kdd.isti.cnr.it/~trasarti/ R. Trasarti (KDD)]:   
 
This very detailed step-by-step procedure was produced by [http://www-kdd.isti.cnr.it/~trasarti/ R. Trasarti (KDD)]:   
  
−
# Install Eclipse
+
# Install [http://www.eclipse.org Eclipse]
 
# Open Eclipse, Skip to 4 if Maven is already installed (last versions of Eclipse have it)
 
# Open Eclipse, Skip to 4 if Maven is already installed (last versions of Eclipse have it)
 
# Copy the address for maven installation: http://download.eclipse.org/technology/m2e/releases
 
# Copy the address for maven installation: http://download.eclipse.org/technology/m2e/releases
Line 21: Line 21:
 
# Implement your algorithm following the template (see attached files)
 
# Implement your algorithm following the template (see attached files)
 
# Open cfg folder in the project and add in transducerers.properties add “APP=app.template.App”
 
# Open cfg folder in the project and add in transducerers.properties add “APP=app.template.App”
−
# Run Test_App.
+
# Run [[#Test_App.java | Test_App]].
  
−
In the template there is an example of how to call a python script during the process procedure.
+
In the [[#App.java | template]] there is an example of how to call a python script during the process procedure.
  
 
== App.java ==
 
== App.java ==
Line 54: Line 54:
  
 
@Override
 
@Override
−
public void init() throws Excepation {
+
public void init() throws Excepa]tion {
 
//IF YOU NEED TO DO SOMETHING BEFORE THE EXECUTION.
 
//IF YOU NEED TO DO SOMETHING BEFORE THE EXECUTION.
 
 

Latest revision as of 17:03, 26 May 2016

This very detailed step-by-step procedure was produced by R. Trasarti (KDD):

  1. Install Eclipse
  2. Open Eclipse, Skip to 4 if Maven is already installed (last versions of Eclipse have it)
  3. Copy the address for maven installation: http://download.eclipse.org/technology/m2e/releases
  4. In Eclipse, install the new plugin (help->install new Software). It takes minutes to load the dependencies.
  5. Create a new project: File->New->Other->Maven->Maven Project
  6. Select the default project type ([…]Quickstart 1.1)
  7. Specify as GroupID = “app”, ArtifactID = “template” (important: Lower Camel Case)
  8. Open pom.xml, click on pom.xml tab (below)
  9. go to: https://wiki.gcube-system.org/gcube/How-to_Implement_Algorithms_for_the_Statistical_Manager
  10. copy the dependencies in the first box under Maven coordinates
  11. paste them into the pom.xml (under dependecies after the existing ones)
  12. go to: https://wiki.gcube-system.org/gcube/Talk:Creating_gCube_Maven_components:_How-To
  13. copy all the setting box
  14. go to your home and move to “.m2/“ folder than create the file settings.xml and paste the text in it. The system will download the dependencies.
  15. download http://goo.gl/aWlPV5
  16. Extract and copy the inner cfg folder (there are two with the same name), and paste the folder in the project
  17. delete the content of the cfg/dynamictransducerers.properties file. save and close it.
  18. Implement your algorithm following the template (see attached files)
  19. Open cfg folder in the project and add in transducerers.properties add “APP=app.template.App”
  20. Run Test_App.

In the template there is an example of how to call a python script during the process procedure.

App.java

package app.template;
 
import java.io.BufferedReader;
import java.io.InputStreamReader;
 
import org.gcube.dataanalysis.ecoengine.datatypes.PrimitiveType;
import org.gcube.dataanalysis.ecoengine.datatypes.StatisticalType;
import org.gcube.dataanalysis.ecoengine.datatypes.enumtypes.PrimitiveTypes;
import org.gcube.dataanalysis.ecoengine.interfaces.StandardLocalExternalAlgorithm;
 
/**
 * Hello world!
 *
 */
public class App extends StandardLocalExternalAlgorithm{
 
	public static final String test_string_name="test_string";
	public static final String resultKey = "result";
	private String result = null;
 
	@Override
	public String getDescription() {
		//RETURN A STRING WITH THE DESCRIPTION OF YOUR ALGORITHM
		return null;
	}
 
	@Override
	public void init() throws Excepa]tion {
		//IF YOU NEED TO DO SOMETHING BEFORE THE EXECUTION.
 
	}
 
	@Override
	protected void process() throws Exception {
 
		//READ THE INPUTS
		String test_string = getInputParameter(test_string_name);
		System.out.println("Input: "+test_string);
 
		//YOUR ALGORITHM MUST BE HERE!
		result = test_string+" (OK)";
 
		//<== START - EXAMPLE OF HOW TO CALL A PHYTON SCRIPT ==>
//		String pythonCmd = getInputParameter("python_location");   
//		ProcessBuilder pb = new ProcessBuilder(pythonCmd, "param1", "param2", ...);
//		Process pr = pb.start();
//
//		BufferedReader stdOutput = new BufferedReader(new InputStreamReader(pr.getInputStream()));
//	    BufferedReader stdError = new BufferedReader(new InputStreamReader(pr.getErrorStream()));
//	    String stdReader = null;
//	    while((stdReader = stdOutput.readLine())!=null) {
//	        System.out.println(stdReader);
//	    }
//	    while((stdReader = stdError.readLine())!=null) {
//	        System.out.println(stdReader);
//	    }
		//<== END - EXAMPLE OF HOW TO CALL A PHYTON SCRIPT ==>
 
	}
 
	@Override
	protected void setInputParameters() {
		//YOU MUST DEFINE YOUR INPUTS
		addStringInput(test_string_name, "Description", "DefaultInput");
	}
 
	@Override
	public void shutdown() {
		//SOME OPERATION AT THE END OF THE PROCESS		
	}
 
 
	@Override
	public StatisticalType getOutput() {
	   PrimitiveType val = new PrimitiveType(String.class.getName(), result , PrimitiveTypes.STRING, resultKey, "This is the result of my algorithm");
	   System.out.println("Output: "+result);
	   return val;
	}
 
 
}

Test_App.java

package app.template;
 
import java.util.List;
 
import org.gcube.dataanalysis.ecoengine.configuration.AlgorithmConfiguration;
import org.gcube.dataanalysis.ecoengine.interfaces.ComputationalAgent;
import org.gcube.dataanalysis.ecoengine.processing.factories.TransducerersFactory;
import org.gcube.dataanalysis.ecoengine.test.regression.Regressor;
 
public class Test_App {
 
	private static AlgorithmConfiguration testConfigLocal() {
 
		AlgorithmConfiguration config = Regressor.getConfig();
		 config.setAgent("APP");
		 config.setParam(App.test_string_name, "HELLO WORLD?");
		 //config.setParam("python_location", "/Library/Frameworks/Python.framework/Versions/2.7/bin/python";);
 
		return config;
	}		
 
	public static void main(String[] args) throws Exception {
		List<ComputationalAgent> trans = null;
		trans = TransducerersFactory.getTransducerers(testConfigLocal());
		trans.get(0).init();
		Regressor.process(trans.get(0));
		trans.get(0).getOutput();
		trans = null;
		}	
 
}
Retrieved from "https://wiki.d4science.org/index.php?title=SoBigData:_step-by-step_procedure_for_algorithm_integration&oldid=21564"