Software Testing – Types

There are many (a lot!) different types of software testing.  I won’t go into any of them in detail, but just wanted to familiarize myself with some of the terms and the basics.

  1. Installation testing – this ensures that the system is installed correctly and working properly at the user’s hardware location.
  2. Compatibility testing – this checks for correct operation between application software when systems or environments are upgraded from the original, possibly causing other pieces to not work correctly.
  3. Smoke testing – this is a minimal attempt to operate the software and is used to determine if there are any basic problems.
  4. Sanity testing – this determines if it is reasonable to go ahead with further (more in-depth) testing.
  5. Regression testing – this focuses on finding defects after a major code change has happened.
  6. Acceptance testing – this may be performed by the customer and is part of the hand-off process between phases of development.
  7. Alpha testing – this is a simulated or actual operational test by potential users
  8. Beta testing – this comes after alpha testing and may be a form of external user acceptance testing.
  9. Functional vs. non-functional testing – this is an activity to verify a specific action or function of the code (does this particular feature work?).
  10. Destructive testing – this attempts to force the software to fail and verifies that the software functions properly even when receiving unexpected inputs.
  11. Performance testing – this is used to determine how a system performs under specific workloads (responsiveness, large quantities of data, large number of users, etc), and there’s a whole set of sub-tests (load, volume, stress, stability – and sometimes these terms are used interchangeably).
  12. Usability testing – this looks at the user interface to see if it is easy to use and understand.
  13. Accessibility testing – this relates to standards associated with the Americans with Disabilities Act of 1990, Section 508 Amendment to the Rehabilitation Act of 1973, and the Web Accessibility Initiative (WAI)
  14. Security testing – checking for security!
  15. Internationalization and localization – this looks at translating software into different languages, keyboards, fonts, bi-directional text, and date/time formats.
  16. Development testing – this supports QA testing and is executed to eliminate construction errors prior to code going to QA.
  17. A/B testing – this is a comparison of two outputs, usually when only one variable has changed, typically used in small-scale situations.
  18. Concurrent testing – this focuses on the performance when running continuously with normal inputs.
  19. Conformance testing – this verifies that a product performs according to its standards.

Whew – what a list!  I really didn’t realize all of the different types of testing.  I think I intuitively have done some of these things before, but didn’t know that it had a specific name.

Software Testing – Levels

I wanted to continue exploring software testing.  This post will focus on levels of testing.  I’ve seen these terms used in different job descriptions that I have come across recently and I wanted to know a little more about them.  From what I have read, there are four levels of testing:

  • Unit testing
  • Integration testing
  • System testing
  • Acceptance testing

Unit testing can also be known as component testing.  This type of testing verifies the functionality of specific pieces of code, particularly at the function level.  Do the functions work the way they are supposed to?  If you are calculating sales tax, is the math correct?  Is it grabbing the right multipliers?  Is the answer in the correct format?  Does the function produce the expected result?

I found that unit tests can be written by the developers as they are working on the code (this was referred to as ‘white-box style’).  This type of testing focuses on the building blocks of the code and does not include whether all the building blocks work well together (this is leading to integration testing).  Unit testing is usually performed by the software developer during the construction phase of the software development life cycle (SDLC).  Instead of sending code to QA to test and then return to the developer multiple times (this does not seem very efficient), the developer will perform tests during construction in order to eliminate basic errors before the larger package of code is sent to QA.

Integration testing is the next step up from unit testing.  It involves testing multiple components in a software product to make sure that they work well together.  Some components may be tested and then additional components added to increase the level or depth of the tests.  This is an iterative approach.  Or you could use the ‘big bang’ approach where all components are lumped together for testing all interfaces and all levels of integration at once.  It sounds like the iterative approach is favored, since it would be easier to identify issues among a smaller number of components.

System testing is also known as ‘end-to-end testing’ and involves tests over the entire software product or system.  This level of testing looks at how the overall product works as well as checking to make sure that the software product does not interfere with the operating environment or other processes within that environment.  Memory corruption, high loads on servers, and slow response times could be indications of the software product interfering with the overall operation environment.

Acceptance testing occurs when the system is delivered to the user and determines whether the user (or customer/client) ‘accepts’ the system.

I know this may seem like a basic topic to some, but it has really been helpful for me to research and learn about these terms and concepts.  Now, when I read articles or job descriptions, I’ll have a much better idea of what they are talking about.

Co-op Summary – Fall Semester 2013

Co-op fall semester summary

Student Name: Kimberly J. Philpot
Major(s): Computer Programming and Database Management
Semester / Year: Fall 2013
Company Name: Vineyard Cincinnati
Supervisor Name: Connie Neckers

This is my Co-op Summary for Fall Semester 2013. In a way, it’s been a long 15 weeks, but also a short 15 weeks. At week 1, I identified several Learning Outcomes with my Co-op Supervisor to work towards during this semester. One of the Learning Outcomes included utilizing my blog, http://www.kimphilpot.wordpress.com, to document my experiences during the co-op. This is an online journal where I posted weekly updates regarding projects I worked on, things I learned ‘on the job,’ and summaries of different research topics. I also invited feedback and comments from my Co-op Supervisor, Connie Neckers, who responded with support and encouragement at every step.

One of the other Learning Outcomes included respecting project deadlines and working in partnership with the IT team to complete projects within the required timeframe. One of the projects I worked on included creating relationship and workflow diagrams for the database, servers, and tasks associated with each server. This was early on in the semester and was a good experience for me to see the visual picture of the relationships between the development server and the production server, including how the testing or QA server relates to both. I included the different server names, the common names for those servers, and their functions. This was also very interesting as it really helped me to see both the server name and the common name together. It also was helpful to see the process. I mistakenly had thought that code sitting in the QA server went directly into Production. But that is not the case. The code is first checked out from the Repository and put into QA for testing. Once it passes QA testing, then the original code from the Repository is put into Production. I always thought that the code moved from QA testing into Production.

Another project I worked on was the mapping of relationships between tables in the Vine database. This required researching the table relationships in the Schema Spy and then creating EER Model diagrams from MySQL showing the relationships between the tables. One of the folks in IT has used Schema Spy to do this initially and it is great, but there are a couple of tables that pretty much everything else relates to, so it is hard to visually see the relationships to the other tables in Schema Spy. The EER model will allow me to print large diagrams (34 x 44 inches) and enable me to spread out the different tables, providing a cleaner view of the relationships.

In addition, another goal for this semester was to be an active participant in the Database Architect meetings and other IT meetings (such as the Google Apps for Enterprise meeting) and provide feedback and thoughts as appropriate. I believe I accomplished this, in that I have heard several times from different IT team members that they appreciated my perspective and comments during meetings. Another goal that I have for this semester is to study and schedule the SQL Server 2012 certification test. Although I think I have gotten a little behind on this item, I do have the test scheduled for the first week of January, so I plan to use my 2 weeks between semesters to study hard for this certification. Earlier in the semester I identified a set of tutorial videos geared specifically for learning SQL 2012 and passing the certification test, so I plan to dig in to those videos and my SQL Server 2012 Training Kit in depth over the next 2 weeks.

An ongoing part of my co-op has been to make periodic website updates for our sister organization at http://www.healingcentercincinnati.org. I have continued to do this and provide great customer service with quick turn-around times. With severe weather knocking at the door, specifically this past week, the need for activating the ‘weather closure’ banner on the website leapt to the top of the list. In the event the organization would close for services, the ‘weather closure’ banner would be activated on the website and social media (Facebook and Twitter) messages would be posted to let people know of the closure. Although the organization did not actually close, I needed to be prepared to make this update outside of normal business hours. Providing great customer service, even to internal customers such as other departments, is critical in that it allows those other departments, in turn, to provide great customer service to their customers.

One last outcome that we identified in Week 1 of the semester was to possibly document the process for printing volunteer labels for Healing Center volunteers. Although I did not formally write about this process, I did talk about it briefly with my Co-op Supervisor. The idea for this particular project started almost 1 year ago, so documenting it now is a little harder.

*****************

Additionally, I have been able to help with several MySQL queries to retrieve specific sets of data or update records from our database. This was a great experience as it increased my hands-on knowledge of MySQL and some of the syntax particulars required for MySQL queries (vs SQL queries). Because I was also in a PHP/MySQL class this semester, it was a good tie-in to help with these MySQL queries in the workplace.

During this past semester, I also noticed that the Software Development Life Cycle (SDLC) was mentioned multiple times, so I did some research to further enhance what I had learned from a previous class the previous semester. The SDLC (Software Development Life Cycle) is the full process, from start (initial idea) to finish (launch and ongoing maintenance) of developing a software program. I would bet that some people lump all of the phases into one or two and probably think that coding is the most important phase. I found some information at http://bpmgeek.com/blog/software-development-life-cycle-basics which was very helpful – the information was clear and easy to understand.

This website (bpmgeek.com) listed 7 steps to the SDLC.

  • Preliminary Idea & Investigation
  • Feasibility Study
  • Analysis
  • Design
  • Coding
  • Testing
  • Maintenance

After a little more searching, I realized that the SDLC can be quite simple (idea, design, develop, release, maintain) to quite complex (showing 13 or more basic phases with multiple sub-phases under each one). As I see it, being able to articulate the SDLC to management or other involved parties would be very helpful. I imagine that there may be a vague understanding of the SDLC, thinking that it is a lot of coding and then magically producing a product that does every single thing the client wants. Obviously, that’s a pretty simple view. But, I’m sure that in some organizations, management folks might not realize all of the details that go into developing software. And those details affect expenses (both at the front end of development time and at the back end when the programmer is correcting things due to poor design or poor articulation of software requirements), client satisfaction, reporting, and ability to do business.

For my own purposes, I developed a hybrid model of the SDLC with the following elements:

  • Idea & Investigation: At Stage 1, Idea & Investigation, the initial idea about the project is identified. Client requirements are investigated and documented. Investigation could also include research about already-existing solutions or modifications to current projects that would meet the needs of the client.
  • Feasibility Study & Analysis: At Stage 2, Feasibility Study & Analysis, the client requirements are analyzed, cost projections are developed, and the full scope of the project is planned out. If there are other already-existing possibly solutions, those are studied as well to determine if that would be the best course of action.
  • Design Specifications: At Stage 3, Design Specifications are developed. This is important, as it will impact the overall cost of the project. Diagrams are developed to visually display the flow of logic within the software. This becomes the blueprint for the software project and needs to happen prior to any coding.
  • Coding & Testing: At Stage 4, Coding & Testing begins. Using the Design Specifications as a blueprint, each section of the software is coded and connected together to develop the overall software project. This stage also includes testing each section to determine if it functions as planned. At this point, any defects that are identified can be corrected.
  • Implement Systems & Evaluate Performance: At Stage 5, Implement Systems & Evaluate Performance, the program is launched and ready to use in the real-world application. Studies are completed to evaluate the performance. Is it operating the way the client wanted? Are there any unexpected results?
  • Ongoing Support & Maintenance: At Stage 6, Ongoing Support & Maintenance, any modifications that arise are made. Enhancements to the program are also included in this phase.

Although the SDLC runs in a clockwise fashion (according to my circular graph), at any stage, it may be necessary to reverse the path and go back to the previous stage to further develop or refine the project. Again, this is more for my personal use to provide a helpful reminder of the different stages in the Software Development Life Cycle.

Although the topic of ‘end user experience’ does not include any coding, it does help to cement the fact that computer programmers and web developers really need to work with the end user in mind. Any program or website needs to be user-friendly, that is, intuitive and easy for the end user to use. Components in a website or web application should be clearly labeled and easy to see for users. One example that I’ve noticed with Google Mail (for our organization) recently is that there is a little grid of squares in the upper right corner – this is not labeled at all. But, when you click on it, the window opens and you can see links for Calendar, Documents, Drive, and other options. Now that I know it’s there, I don’t need it labeled, but when it first appeared, it would have been nice to have a hover balloon pop up to let you know what the little grid of squares is. This is just an example of how web developers could enhance their web pages to provide a great experience for the end users.

I think the opportunity and requirement to document my learning experiences has been the most helpful over the last 15 weeks. It is important to be able to articulate what you know to others, and to be able to share your knowledge with others. Overall, my co-op experience has been a positive experience and I look forward to continuing this in the next semester.

Co-op Highlights 11

Last week, I was able to explore some of the controls/extenders in the Ajax toolkit in ASP.NET.  The Twitter control was fairly easy, but still somewhat challenging.  I needed to get the Consumer Key, Consumer Secret, Access Token, and Access Token Secret from the Twitter API and put them in the web.config file.  Fortunately, Twitter makes it very easy to do this – https://dev.twitter.com/apps/new.

I also found this website that talks about each one of the controls – http://www.ajaxcontroltoolkit.com/. But, it still lacked a little detail on some of them (for me anyways). I found a website (possibly Stack Overflow) that gave a clue of adding ‘act:’ before each of the Keys/Secrets and that did the trick.  It’s a small thing I had to do for another class, but it allowed me the opportunity to see how the control worked.

Our sister organization, Company B, is beginning to undergo a remodel of their website.  This is exciting as I will be somewhat involved with the Design Specification (back to the SDLC) phase and helping with some of the coding as time allows.  Just maybe, I’ll get a chance to implement the Twitter Ajax control on this site – or at least offer it.  Currently, there is just a link to Company B’s Twitter page, but not a complete feed (like the Ajax control shows).