Difference between revisions of "Procedure QA Deployment"
Andrea.manzi (Talk | contribs) (→Procedures) |
(→Understanding the packages to upgrade) |
||
| (16 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
== QA Deployment == | == QA Deployment == | ||
| − | One of the phase of the software lifecycle is the validation of the software in an environment production-like. This phase is also referred | + | One of the phase of the software lifecycle is the validation of the software in an environment production-like. This phase is also referred as pre-production or QA deployment and validation and it has been implemented as well in the case of gCube and its deployment on the D4Science infrastructure. |
| − | Given the limited resources and effort it was not possible to replicate an environment as the production one for the QA phase, therefore part of the resources exploited during this phase are shared with the production environment. In the case of gCube the list of the resources dedicated to QA are available on the [http://wiki. | + | Given the limited resources and effort it was not possible to replicate an environment as the production one for the QA phase, therefore part of the resources exploited during this phase are shared with the production environment. In the case of gCube resources the list of the resources dedicated to QA are available on the [http://wiki.d4science.org/index.php/Pre-Production_Resources wiki page]. As described on the wiki 2 sites are contributing : NKUA and CNR. |
| − | This sharing with the production infrastructure is also reflected at the level of scope hierarchy: the QA environment is not a separated infrastructure as it happens for the development and the testing environment but it consists | + | This sharing with the production infrastructure is also reflected at the level of scope hierarchy: the QA environment is not a separated infrastructure as it happens for the development and the testing environment but it consists of a VO ( '''Ecosystem''') under the production infrastructure scope. (''/d4science.research-infrastructures/Ecosystem''). One VRE is also available ('''TryIt''') in order to validate VRE specific services and portlets |
=== Procedures === | === Procedures === | ||
| − | As already said the QA phase has limited resources allocated therefore its focus is on a subset of the upgrade activities described | + | As already said the QA phase has limited resources allocated therefore its focus is on a subset of the upgrade activities described on the Production upgrade procedure [http://wiki.d4science.org/index.php/Procedure_Infrastructure_upgrade#Upgrade_Activities wiki]: |
* testing the upgrade of the deployed containers using the scripts | * testing the upgrade of the deployed containers using the scripts | ||
| Line 14: | Line 14: | ||
* testing the deployment of the portal components ( on a dedicated pre-production portal) | * testing the deployment of the portal components ( on a dedicated pre-production portal) | ||
| − | in addition this phase include | + | in addition this phase include validation of the deployment components with the data present in production. |
For the software upgrade activities the QA validation shares together with the testing phase the maven repository in order to install the components: | For the software upgrade activities the QA validation shares together with the testing phase the maven repository in order to install the components: | ||
| Line 20: | Line 20: | ||
* http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-staging/ | * http://maven.research-infrastructures.eu/nexus/content/repositories/gcube-staging/ | ||
| − | As described on the [http://wiki. | + | As described on the [http://wiki.d4science.org/index.php/Procedure_Infrastructure_upgrade#Upgrade_Activities Upgrade Activities] section this is required in order to download the containers ( gHN and SmartGears), the gCube Service full gars, the portlets and the rest of the portal components. The repository is also used in the case of Dynamic Deployment to download the artifacts: for this purpuse a dedicated instance of the Software Gateway service is deployed on the Ecosystem VO. This means that when a new release has to be validated on the QA environment the Software Gateway has to be populated. |
| − | Usually the QA phase starts when the | + | Usually the QA phase starts when the integration phase is almost completed: this is true when most of the gCube components correctly builds and the deployment tests are fine. |
| + | Once started the QA phase tries to complete the software deployment and validation as fast as possible in order to bring the new software in the production environment and to the user, trying on the other hand to cover as mush as possible use cases ( in order to avoid buggy software to reach the end-user). In general a week of work is scheduled for the QA validation but this varies according to the size of the release and the number of issues found during this phase that have then to be solved by the developers | ||
| − | + | As already described in the [http://wiki.d4science.org/index.php/Procedure_Infrastructure_upgrade#Preparation_of_the_upgrade_plan gCube upgrade page] the activities to be performed have to be listed on the http://wiki.d4science.org/index.php/Resources_Upgrade Upgrade page] , in particular the section with scope Ecosystem refers to the QA environment. This page is also input for the future production deployments cause most of the activities described for the Ecosystem VO will have to be applied for the production VOs as well, in particular the portal upgrade. | |
| − | + | The next section together with the wiki on the [http://wiki.d4science.org/index.php/Procedure_Infrastructure_upgrade infrastructure upgrade] are a guide the QA upgrades in other to understand what has to be upgraded and how. | |
| − | == | + | == Understanding the packages to upgrade == |
| − | A gCube release result of the integration activity | + | A gCube release result of the integration activity could include a high number of new and upgrade components. The information about the components that are part of a release are listed in Redmine under the report ( tracker Release) |
| − | https:// | + | https://support.d4science.org/projects/gcube/issues?set_filter=1&tracker_id=6 |
Each subsystem listed on the report includes different types of components to upgrade/install on the infrastructure. In order to understand what are the components to include and how to plan the upgrade we have a list of known categories. | Each subsystem listed on the report includes different types of components to upgrade/install on the infrastructure. In order to understand what are the components to include and how to plan the upgrade we have a list of known categories. | ||
| Line 66: | Line 67: | ||
In addiction to these subsystem, some other GUI side components are part of the subsystems. org.gcube.application and org.gcube.messaging. | In addiction to these subsystem, some other GUI side components are part of the subsystems. org.gcube.application and org.gcube.messaging. | ||
| − | == | + | === Sources of information === |
| − | In | + | In ETICS we have defined different numbering for each component released: |
x.y.z-i | x.y.z-i | ||
| Line 76: | Line 77: | ||
y = minor : if the y version change, the component has been enhanced but the changes are all backward compatible | y = minor : if the y version change, the component has been enhanced but the changes are all backward compatible | ||
| − | + | z = patch : if the z version change, the components includes a fix only | |
i = build : we use this version change only to track changes at the level of packaging/etics. The components has not changes at the level of code or configuration so no upgrade should be performed. | i = build : we use this version change only to track changes at the level of packaging/etics. The components has not changes at the level of code or configuration so no upgrade should be performed. | ||
| + | |||
| + | It's required a knowledge of the gCube software to understand the relation between the packages released and the software installed. During this phase it's also requested a collaboration with the development team in order to better understand what are the changes introduced by the released components. In addition some source of inputs are: | ||
| + | |||
| + | * Redmine: (https://support.d4science.org/projects/gcube/issues): The tickets related to the milestone to be installed in QA are quite important to understand what are the changes introduced in the release. | ||
| + | |||
| + | * Changelogs: this info are collected for each released and extracted using the [http://grids16.eng.it/BuildReport/home/Recent_Builds ETICS BTRT]. This information is quite useful in some cases and it complements the information available on Redmine. An example of Change log report is available at: | ||
| + | |||
| + | http://grids16.eng.it/BuildReport/bdownload/Recent_Builds/org.gcube.3-1-0/BUILD_15/reports/distribution/releasenotes.xml | ||
| + | |||
| + | * Information System: In most cases we can use the information available on the Information system to understand the version of the components deployed in a given container. The infrastructure [https://services.d4science.org/web/guest/monitor monitor] gives information not only on the available services running on the infra, but also on the version installed. This can be used on the upgrade phase to understand which services has to be upgraded and where. | ||
Latest revision as of 12:55, 8 September 2015
QA Deployment
One of the phase of the software lifecycle is the validation of the software in an environment production-like. This phase is also referred as pre-production or QA deployment and validation and it has been implemented as well in the case of gCube and its deployment on the D4ScienceAn e-Infrastructure operated by the D4Science.org initiative. infrastructure.
Given the limited resources and effort it was not possible to replicate an environment as the production one for the QA phase, therefore part of the resources exploited during this phase are shared with the production environment. In the case of gCube resources the list of the resources dedicated to QA are available on the wiki page. As described on the wiki 2 sites are contributing : NKUA and CNR. This sharing with the production infrastructure is also reflected at the level of scope hierarchy: the QA environment is not a separated infrastructure as it happens for the development and the testing environment but it consists of a VOVirtual Organization; ( Ecosystem) under the production infrastructure scope. (/d4science.research-infrastructures/Ecosystem). One VREVirtual Research Environment. is also available (TryIt) in order to validate VREVirtual Research Environment. specific services and portlets
Procedures
As already said the QA phase has limited resources allocated therefore its focus is on a subset of the upgrade activities described on the Production upgrade procedure wiki:
- testing the upgrade of the deployed containers using the scripts
- testing the deployment of new developed gCube services
- testing the deployment of the portal components ( on a dedicated pre-production portal)
in addition this phase include validation of the deployment components with the data present in production.
For the software upgrade activities the QA validation shares together with the testing phase the maven repository in order to install the components:
As described on the Upgrade Activities section this is required in order to download the containers ( gHN and SmartGears), the gCube Service full gars, the portlets and the rest of the portal components. The repository is also used in the case of Dynamic Deployment to download the artifacts: for this purpuse a dedicated instance of the Software Gateway service is deployed on the Ecosystem VOVirtual Organization;. This means that when a new release has to be validated on the QA environment the Software Gateway has to be populated.
Usually the QA phase starts when the integration phase is almost completed: this is true when most of the gCube components correctly builds and the deployment tests are fine. Once started the QA phase tries to complete the software deployment and validation as fast as possible in order to bring the new software in the production environment and to the user, trying on the other hand to cover as mush as possible use cases ( in order to avoid buggy software to reach the end-user). In general a week of work is scheduled for the QA validation but this varies according to the size of the release and the number of issues found during this phase that have then to be solved by the developers
As already described in the gCube upgrade page the activities to be performed have to be listed on the http://wiki.d4science.org/index.php/Resources_Upgrade Upgrade page] , in particular the section with scope Ecosystem refers to the QA environment. This page is also input for the future production deployments cause most of the activities described for the Ecosystem VOVirtual Organization; will have to be applied for the production VOs as well, in particular the portal upgrade.
The next section together with the wiki on the infrastructure upgrade are a guide the QA upgrades in other to understand what has to be upgraded and how.
Understanding the packages to upgrade
A gCube release result of the integration activity could include a high number of new and upgrade components. The information about the components that are part of a release are listed in Redmine under the report ( tracker Release)
https://support.d4science.org/projects/gcube/issues?set_filter=1&tracker_id=6
Each subsystem listed on the report includes different types of components to upgrade/install on the infrastructure. In order to understand what are the components to include and how to plan the upgrade we have a list of known categories.
Bundle components
Belongs to this group of components the base containers of our infrastructure components:
- org.gcube.portal.portal-bundle
- org.gcube.distribution.ghn-distribution
- org.gcube.distribution.smartgears-distribution
When a new version of these components is released we should check the impact in terms of libraries/configuration to upgrade and prepare upgrade scripts/detailed instructions.
Enabling Layer components
Belongs to this category the components that are the the enablers of our infrastructure, the base services that allows discovery, resource management, service deployments etc etc:
- org.gcube.vre-management
- org.gcube.information-system
- org.gcube.vo-management
- org.gcube.messaging
- org.gcore
Portal components
Components GUI side to to be installed only on the infrastructure portal:
- org.portlets-*
- org.portal-
- org.gcube.application-support-layer.*
In addiction to these subsystem, some other GUI side components are part of the subsystems. org.gcube.application and org.gcube.messaging.
Sources of information
In ETICS we have defined different numbering for each component released:
x.y.z-i
x = major : if the x version of the component has changed compared to the previous release, therefore his public Interface has changed, so its upgrade depends on the availability of the other components to adapt to it
y = minor : if the y version change, the component has been enhanced but the changes are all backward compatible
z = patch : if the z version change, the components includes a fix only
i = build : we use this version change only to track changes at the level of packaging/etics. The components has not changes at the level of code or configuration so no upgrade should be performed.
It's required a knowledge of the gCube software to understand the relation between the packages released and the software installed. During this phase it's also requested a collaboration with the development team in order to better understand what are the changes introduced by the released components. In addition some source of inputs are:
- Redmine: (https://support.d4science.org/projects/gcube/issues): The tickets related to the milestone to be installed in QA are quite important to understand what are the changes introduced in the release.
- Changelogs: this info are collected for each released and extracted using the ETICS BTRT. This information is quite useful in some cases and it complements the information available on Redmine. An example of Change log report is available at:
http://grids16.eng.it/BuildReport/bdownload/Recent_Builds/org.gcube.3-1-0/BUILD_15/reports/distribution/releasenotes.xml
- Information System: In most cases we can use the information available on the Information system to understand the version of the components deployed in a given container. The infrastructure monitor gives information not only on the available services running on the infra, but also on the version installed. This can be used on the upgrade phase to understand which services has to be upgraded and where.
