R algorithm integration with Statistical Manager
From D4Science Wiki
Hypothesis and Thesis
The SDMX Data converter is part of serie of experiments to test and assess how data managers / developers can plug easily algorithms (especially R algorithms) in the infrastructure, through the Statistical Manager tool, and respond quickly to data analysis needs while benefiting of iMarine computing resources.
The product of this experiment is a service that allows to convert a SDMX dataset, provided through a SDMX service URL, to the CSV format.
The broader scope of this experiment is:
- to assess how a data manager / developer can plug an algorithm by their own,
- to identify potential improvements to make the R script integration quick and easy
Outcome
TBD
Activity Workflow
- The activity was done by familiarizing with the Statistical Manager, relying both on the documentation and a tutorial made available to facilitate the integration of algorithms.
- A basic R script was created to test the Statistical Manager. This script allows to convert a SDMX-ML dataset to CSV.
- In order to integrate the R script, a separate Java Maven project was created (with the aim to add further algorithm later).
- Few exchange with the Statistical Managers developers was required for the project settings, an highlighted some few scatter in the documentation
- The R script was integrated in the project, tested and sent to Statistical Manager team for its deployment
- Additional exchange with the team took place, to have some clarifications on:
- algorithms inputs (difference between a File input and remote resource - URL - input)
- the need for data managers to indicate the eventual R package dependencies to install prior to the algorithm deployment
- how to add the algorithm within a given category of algorithms (for display purpose)
- The algorithm was successfully deployed and is currently operational in the development portal
Conclusion
TBD
Recommendations & future developments
TBD
Experimentation
TBD
