What is a vCenter plugin?
vCenter plugins are additional features of the vCenter web client that aid in the management of vendor specific operations (on Storage, Networking and virtualization). These plugins are developed by third party developers to provide a vendor specific user interface in vCenter. For example, a storage vendor can extend the vCenter web client interface to provide additional information about inventory, hosts, virtual machines and statistics about CPU and memory utilization with vCenter Plugin.
What is Automation Framework ?
Automation Framework is the framework that provides an environment to input test data, automate execution of test cases and generation of execution reports. The automation framework must contain the below components:
- Test Data – Test data can be inputted to the automation framework from external data sources like excel file, properties file, database, xml files etc.
- Test Execution – Test execution logic written using a testing tool like Selenium that reads test data and execute the test cases.
- Test Reports – Generation of reports that shows overall statistics of test cases executed.
What are the types of automation framework?
There are different types of automation frameworks, but I will look at only the most used approaches and their requirements:
- Data Driven Testing Framework: This framework is used when one has to test the same functionality with different sets of input data. In this framework, the input data is an external database (Excel or XML file) and the test script is written which executes repeatedly for each set of input in the test database.
For example, testing a login form with different set of username and password combinations
Pros: Reduces code for test script as all possibilities of inputs are tested with same test script.
Cons: Knowledge of programming language is a must. Extra efforts required to store and read the data from external database - Keyword Driven Testing Framework: Keyword driven testing is an extension of the data driven testing framework. In this framework, it is imperative to identify the web elements in the application under test and then derive list of keywords i.e actions. Keywords are nothing but methods/functions in the test script which will read the data from external database (example excel file) and perform the necessary action. The test data in file includes keyword, input data, locators and expected data.
- Keywords – can be like click, sendKeys, OpenBrowser, select, wait etc.
- Input data – The text needs to inputted to the field else it will be blank.
- Locators – web element locators such as id, name & xpath.
- Expected data – used to verify the link or labels in web UI.
Pros: User only needs the knowledge of keywords and doesn’t need to know how to script.The same keywords can be used multiple times in test cases for different applications.
Cons: If the number of keyword increases, the framework becomes more complicated. - Hybrid Testing Framework: Hybrid Test Framework is a combination of one or more automation framework like the above. The purpose is to leverage the benefits of different automation frameworks together.
Fig 1.1 Test Framework Structure
What is test automation for VCP?
Test Automation for VCP is a test automation framework for vCenter plugin for ESXi 6.0 and onwards. It is the hybrid framework to cater to the GUI aspect of the vCenter plugin, vCenter REST API and Command Line Interface. Automation for the VCP plugin is done from the Installation phase through to functional testing of GUI, use of Rest API and CLI. In VCP automation, both data driven and keyword driven frameworks are used. Test data is managed in Excel files, such as driver file, test case files and installer script file. The test execution report is generated ina HTML file once the test is completed. These report shows the time the test took, number of test cases, number of total test steps, pass and fail tests, percentage of passing and reason of failure etc.Technology used – java 1.8, Extent, Selenium, Maven, TestNg, HTML, Jenkins, Install4j and AutoItx4Java.
Provides:
- GUI testing (HTML, Angular)
- Installation and UN-installation of plugin
- VMware rest API automation.
- Command Line Interface automation
- Reporting with extent reports
- Test data in excel format
What are the benefits?
- Minimal manual intervention
- Maximum coverage
- Low cost maintenance
- Easy Reporting
- Reduced Time
What are the challenges?
- Selecting a Proper Testing Approach: While selecting an automation framework, ask some very basic but important questions to arrive at the right methodology:
- Will the framework reduce the time and cost for testing?
- Will the framework useful in future and have a long life?
- Can we add new actions/keywords with minimum changes?
- Can the framework generate useful test reports and metrics?
- Selecting right technology/tool: Since it is a major challenge to find the right technology for the testing framework, you need to try different tools to come up with the one that suits your requirement. For VCP plugin installer automation, we tried many tools like Sikuli, Winimum and AutoIt but finally decided on AutoIT because it can be easily integrated with Selenium.
- Test Case File format: The format of test case file should be optimized so that the end user will have to enter minimum inputs to fill up the test case file.
- Automating Rest API : You need to understand the REST API documentation of the vendor and then decide the rest test case file implementation such that it must –
- Authenticate the rest and getting session. Using same session for the further API calls.
- Support different request types such as GET, PUT, DELETE & POST
- Handling of response and validating with expected results
- Handling Web Elements: If the User Interface for the plugin is in HTML, it is simple to write the test case actions, but if it is built with Angular JS, you need to handle it differently as selenium does not support Angular elements like grid, graph, & fields.
- Generating Test Reports: Default reports are unattractive and information shown is not sufficient for end user. Test Report should show all statistics including – time taken, percentage of pass and failure, reason of failure, platform on which execution is done. To this effect, you use Extent Report which is a third party library which has an integration with Selenium.
- How does it help?
- It validates functional testing as well as installation in an iterative fashion
- It is integrated with CI and REST to validate functional correctness
- It will reduces testing time
- Generate the reports which gives the overall statistics of test Execution
- It can be later extended to VRO too
- How much coverage is possible? Below table shows the coverage:
User Interface Coverage Percentage ESX 6.o 30-40% ESX 6.5 and onwards 80-90%
[Tweet “Basics of vCenter Plugin Automation ~ via @CalsoftInc”]
