R algorithm integration with Statistical Manager

Hypothesis and Thesis

This experiments aims to test and assess how data managers / developers can plug easily algorithms (especially R algorithms) in the infrastructure, through the Statistical Manager tool, and respond quickly to data analysis needs while benefiting of iMarine computing resources.

The product of this experiment is a basic service that allows to convert a SDMX dataset, provided through a SDMX service URL, to the CSV format.

The broader scope of this experiment is:

• to assess how a data manager / developer can plug an algorithm by their own,
• to identify potential improvements to make the R script integration quick and easy

Outcome

TBD

Activity Workflow

• The activity was done by familiarizing with the Statistical Manager, relying both on the documentation and a tutorial video made available to facilitate the integration of algorithms.
• A basic R script was created to test the Statistical Manager. This script allows to convert a SDMX-ML dataset to CSV.
• In order to integrate the R script, a separate Java Maven project was created (with the aim to add further algorithm later).
• Few exchange with the Statistical Managers developers was required for the project settings, an highlighted some few scatter in the documentation
• The R script was integrated in the project, tested and sent to Statistical Manager team for its deployment
• Additional exchange with the team took place, to have some clarifications on:
  • algorithms inputs (difference between a File input and remote resource - URL - input)
  • the need for data managers to indicate the eventual R package dependencies to install prior to the algorithm deployment
  • how to add the algorithm within a given category of algorithms (for display purpose in the Statistical Manager user interface)
• The algorithm was successfully deployed and is currently operational in the development portal

Conclusion

TBD

Recommendations & future developments

TBD

Experimentation

TBD

Related links

• How to implement algorithms for the Statistical Manager
• Implement Statistical Manager algorithms - tutorial video
• rsdmx package for R