Difference between revisions of "Procedure QA Deployment"

From D4Science Wiki
Jump to: navigation, search
(Understanding the packages to upgrade)
(QA Deployment)
Line 1: Line 1:
 
== QA Deployment  ==
 
== QA Deployment  ==
  
One of the phase of the software lifecycle is the validation of the software in an environment production-like. This phase is also referred an pre-production or QA deployment and validation and it has been implemented as well in the case of gCube and its deployment on the D4Science infrastructure.
+
One of the phase of the software lifecycle is the validation of the software in an environment production-like. This phase is also referred as pre-production or QA deployment and validation and it has been implemented as well in the case of gCube and its deployment on the D4Science infrastructure.
  
Given the limited resources and effort it was not possible to replicate an environment as the production one for the QA phase, therefore part of the resources exploited during this phase are shared with the production environment. In the case of gCube the list of the resources dedicated to QA are available on the [http://wiki.i-marine.eu/index.php/Pre-Production_Resources wiki page]. As described on the wiki 2 sites are contributing : NLKUA and CNR.  
+
Given the limited resources and effort it was not possible to replicate an environment as the production one for the QA phase, therefore part of the resources exploited during this phase are shared with the production environment. In the case of gCube resources the list of the resources dedicated to QA are available on the [http://wiki.i-marine.eu/index.php/Pre-Production_Resources wiki page]. As described on the wiki 2 sites are contributing : NKUA and CNR.  
This sharing with the production infrastructure is also reflected at the level of scope hierarchy: the QA environment is not a separated infrastructure as it happens for the development and the testing environment but it consists on a VO  ( '''Ecosystem''') under the production infrastructure scope. (''/d4science.research-infrastructures/Ecosystem''). One VRE is also available ('''TryIt''') in order to validate VRE specific services and portlets
+
This sharing with the production infrastructure is also reflected at the level of scope hierarchy: the QA environment is not a separated infrastructure as it happens for the development and the testing environment but it consists of a VO  ( '''Ecosystem''') under the production infrastructure scope. (''/d4science.research-infrastructures/Ecosystem''). One VRE is also available ('''TryIt''') in order to validate VRE specific services and portlets
  
 
=== Procedures ===
 
=== Procedures ===
  
As already said the QA phase has limited resources allocated therefore its focus is on a  subset of the upgrade activities described at on the Production upgrade procedure [http://wiki.i-marine.eu/index.php/Procedure_Infrastructure_upgrade#Upgrade_Activities wiki]:
+
As already said the QA phase has limited resources allocated therefore its focus is on a  subset of the upgrade activities described  on the Production upgrade procedure [http://wiki.i-marine.eu/index.php/Procedure_Infrastructure_upgrade#Upgrade_Activities wiki]:
  
 
* testing the upgrade of the deployed containers using the scripts
 
* testing the upgrade of the deployed containers using the scripts
Line 14: Line 14:
 
* testing the deployment of the portal components ( on a dedicated pre-production portal)
 
* testing the deployment of the portal components ( on a dedicated pre-production portal)
  
in addition this phase include some validation of the deployment components with the data present in production.
+
in addition this phase include validation of the deployment components with the data present in production.
  
 
For the software upgrade activities the QA validation shares together with the testing phase the maven repository in order to install the components:
 
For the software upgrade activities the QA validation shares together with the testing phase the maven repository in order to install the components:
Line 20: Line 20:
 
* http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-staging/
 
* http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-staging/
  
As described on the [http://wiki.i-marine.eu/index.php/Procedure_Infrastructure_upgrade#Upgrade_Activities Upgrade Activities] section this is required in order to dowload the containes ( gHN and SmartGears), the gCube Sevice full gars, the portlets and the rest of the portal components. The repository is also used in the case of Dynamic Deployment to download the artifacts: for this porpuse a dedicated instance of the Software Gateway service is deployed on the Ecosystem VO. This means that when a new release has to be validated on the QA environment the Software Gateway has to be populated ( this activity is performed by ENG in the context of WP7 and they should be contacted for this).
+
As described on the [http://wiki.i-marine.eu/index.php/Procedure_Infrastructure_upgrade#Upgrade_Activities Upgrade Activities] section this is required in order to download the containers ( gHN and SmartGears), the gCube Service full gars, the portlets and the rest of the portal components. The repository is also used in the case of Dynamic Deployment to download the artifacts: for this purpuse a dedicated instance of the Software Gateway service is deployed on the Ecosystem VO. This means that when a new release has to be validated on the QA environment the Software Gateway has to be populated ( this activity is performed by ENG in the context of WP7 and they should be contacted for this).
  
Usually the QA phase starts when the WP7 integration phase is almost completed: this is true when most of the gCube components correctly builds and the deployment tests are fine. Since it needs as input  the output of the WP7 processes, coordination is very important at this level and for this purpuse members of the WP5 team are in touch during the software integration phase with the members of the WP7 team to understand the status of the activities.
+
Usually the QA phase starts when the WP7 integration phase is almost completed: this is true when most of the gCube components correctly builds and the deployment tests are fine. Since it needs as input  the output of the WP7 processes, coordination is very important at this level and for this purpose members of the WP5 team are in touch during the software integration phase with the members of the WP7 team to understand the status of the activities.
  
Once started the QA phase tries to complete the software deployment and validation as fast as possible in order to bring the new software in  the production environment and to the user as fast as possible, trying on the other hand to cover as mush as use case possible ( in order to avoid buggy software to reach the user). In general a week of work is scheduled for the QA validation but this varies according to the size of the release and the number or issues found during this phase that have then to be solved.
+
Once started the QA phase tries to complete the software deployment and validation as fast as possible in order to bring the new software in  the production environment and to the user, trying on the other hand to cover as mush as possible use cases ( in order to avoid buggy software to reach the end-user). In general a week of work is scheduled for the QA validation but this varies according to the size of the release and the number of issues found during this phase that have then to be solved by the developers
  
As already described in the [http://wiki.i-marine.eu/index.php/Procedure_Infrastructure_upgrade#Preparation_of_the_upgrade_plan gCube upgrade page] the activities to be performed are listed on the http://wiki.i-marine.eu/index.php/Resources_Upgrade Upgrade page] , in particular the section with scope Ecosystem. This page is also input for the future production deployment cause most of the activities described for the Ecosystem VO will have to be applied for the production VOs as well , in particular the portal upgrade.
+
As already described in the [http://wiki.i-marine.eu/index.php/Procedure_Infrastructure_upgrade#Preparation_of_the_upgrade_plan gCube upgrade page] the activities to be performed have to be listed on the http://wiki.i-marine.eu/index.php/Resources_Upgrade Upgrade page] , in particular the section with scope Ecosystem refers to the QA environment. This page is also input for the future production deployments cause most of the activities described for the Ecosystem VO will have to be applied for the production VOs as well, in particular the portal upgrade.
  
 
The next section together with the wiki on the [http://wiki.i-marine.eu/index.php/Procedure_Infrastructure_upgrade infrastructure upgrade] are a guide  the QA upgrades in other to understand what has to be upgraded and how.
 
The next section together with the wiki on the [http://wiki.i-marine.eu/index.php/Procedure_Infrastructure_upgrade infrastructure upgrade] are a guide  the QA upgrades in other to understand what has to be upgraded and how.

Revision as of 08:07, 16 April 2014

QA Deployment

One of the phase of the software lifecycle is the validation of the software in an environment production-like. This phase is also referred as pre-production or QA deployment and validation and it has been implemented as well in the case of gCube and its deployment on the D4ScienceAn e-Infrastructure operated by the D4Science.org initiative. infrastructure.

Given the limited resources and effort it was not possible to replicate an environment as the production one for the QA phase, therefore part of the resources exploited during this phase are shared with the production environment. In the case of gCube resources the list of the resources dedicated to QA are available on the wiki page. As described on the wiki 2 sites are contributing : NKUA and CNR. This sharing with the production infrastructure is also reflected at the level of scope hierarchy: the QA environment is not a separated infrastructure as it happens for the development and the testing environment but it consists of a VOVirtual Organization; ( Ecosystem) under the production infrastructure scope. (/d4science.research-infrastructures/Ecosystem). One VREVirtual Research Environment. is also available (TryIt) in order to validate VREVirtual Research Environment. specific services and portlets

Procedures

As already said the QA phase has limited resources allocated therefore its focus is on a subset of the upgrade activities described on the Production upgrade procedure wiki:

  • testing the upgrade of the deployed containers using the scripts
  • testing the deployment of new developed gCube services
  • testing the deployment of the portal components ( on a dedicated pre-production portal)

in addition this phase include validation of the deployment components with the data present in production.

For the software upgrade activities the QA validation shares together with the testing phase the maven repository in order to install the components:

As described on the Upgrade Activities section this is required in order to download the containers ( gHN and SmartGears), the gCube Service full gars, the portlets and the rest of the portal components. The repository is also used in the case of Dynamic Deployment to download the artifacts: for this purpuse a dedicated instance of the Software Gateway service is deployed on the Ecosystem VOVirtual Organization;. This means that when a new release has to be validated on the QA environment the Software Gateway has to be populated ( this activity is performed by ENG in the context of WP7 and they should be contacted for this).

Usually the QA phase starts when the WP7 integration phase is almost completed: this is true when most of the gCube components correctly builds and the deployment tests are fine. Since it needs as input the output of the WP7 processes, coordination is very important at this level and for this purpose members of the WP5 team are in touch during the software integration phase with the members of the WP7 team to understand the status of the activities.

Once started the QA phase tries to complete the software deployment and validation as fast as possible in order to bring the new software in the production environment and to the user, trying on the other hand to cover as mush as possible use cases ( in order to avoid buggy software to reach the end-user). In general a week of work is scheduled for the QA validation but this varies according to the size of the release and the number of issues found during this phase that have then to be solved by the developers

As already described in the gCube upgrade page the activities to be performed have to be listed on the http://wiki.i-marine.eu/index.php/Resources_Upgrade Upgrade page] , in particular the section with scope Ecosystem refers to the QA environment. This page is also input for the future production deployments cause most of the activities described for the Ecosystem VOVirtual Organization; will have to be applied for the production VOs as well, in particular the portal upgrade.

The next section together with the wiki on the infrastructure upgrade are a guide the QA upgrades in other to understand what has to be upgraded and how.

Components forming a gCube Release

A gCube release result of the integration activity implemented by WP7 could include a high number of new and upgrade components. The information about the components that are part of a release are listed in trac under the report ( grouped by milestone)

https://issue.imarine.research-infrastructures.eu/report/17

Each subsystem listed on the report includes different types of components to upgrade/install on the infrastructure. In order to understand what are the components to include and how to plan the upgrade we have a list of known categories.

Bundle components

Belongs to this group of components the base containers of our infrastructure components:

  • org.gcube.portal.portal-bundle
  • org.gcube.distribution.ghn-distribution
  • org.gcube.distribution.smartgears-distribution

When a new version of these components is released we should check the impact in terms of libraries/configuration to upgrade and prepare upgrade scripts/detailed instructions.

Enabling Layer components

Belongs to this category the components that are the the enablers of our infrastructure, the base services that allows discovery, resource management, service deployments etc etc:

  • org.gcube.vre-management
  • org.gcube.information-system
  • org.gcube.vo-management
  • org.gcube.messaging
  • org.gcore

Portal components

Components GUI side to to be installed only on the infrastructure portal:

  • org.portlets-*
  • org.portal-
  • org.gcube.application-support-layer.*

In addiction to these subsystem, some other GUI side components are part of the subsystems. org.gcube.application and org.gcube.messaging.

Understanding the packages to upgrade

In Etics we have defined different numbering for each component released:

x.y.z-i

x = major : if the x version of the component has changed compared to the previous release, therefore his public Interface has changed, so its upgrade depends on the availability of the other components to adapt to it

y = minor : if the y version change, the component has been enhanced but the changes are all backward compatible

y = patch : if the z version change, the components includes a fix only

i = build : we use this version change only to track changes at the level of packaging/etics. The components has not changes at the level of code or configuration so no upgrade should be performed.

when a new release is scheduled, the packages upgraded/removed/newly added on ETICS are described on the iMarine TRAC through release tickets:

https://issue.imarine.research-infrastructures.eu/report/17

tickets are related to gCube subsystems and it's required a knowledge of the gCube software to understand the relation between the packages released and the software installed. But we can use the information available on the Information system to understand the version of the components deployed in a given container