Team Foundation Server - Tracking & Reporting

Editor's Notes

1. By using Visual Studio Application Lifecycle Management (ALM), you can manage customer needs more effectively. You can create a high-level plan that breaks your project down into potentially shippable increments, and you can create detailed plans to execute shorter iterations in which you develop those increments.Because you develop detailed plans at the start of each iteration, you have more certainty in how the plan progresses with each milestone that you reach. When your team finishes each iteration, you can refine the high-level plan based on what you might have learned during the iteration. You can also replan any work that was not completed.Your team can use Visual Studio ALM and apply an appropriate process template to plan, develop, and track your project iteratively.
2. You can use the Product Planning workbook to manage the backlog and development of user stories, determine the team velocity, and balance the workload across several iterations, also known as sprints. To plan an iteration, you review, rank, prioritize, and assign points to the stories that will be implemented for a project. To balance workload, you assign each story to a specific iteration and adjust these assignments until the number of story points that are assigned across all iterations are roughly equal. Product Backlog: You use this worksheet to filter, rank, and prioritize the user stories that you want to manage. You can specify story points and assign user stories to iterations. The Product Backlog worksheet references the Product Backlog team query, which finds all user stories that are defined for the team project. Within the workbook, you can filter the stories based on product area. Iteration Planning: You use this worksheet to schedule the iterations, review the workload for each iteration, and determine how to balance the workload across the iterations. Interruptions: You use this worksheet to specify holidays or other dates when the team will perform no work.
3. You can generate several reports in Microsoft Excel that show current status and historical data based on the filter criteria that you specify in a flat-list work item query. This is useful to show the distribution of work items according to selected criteria or to view trends for the past several weeks. In addition, it is an effective way for you to quickly generate PivotTable and PivotChart reports that you can customize to support other report views. When you create an Excel report from a query, you can choose which reports to generate based on the variables that are used to filter the query and the criteria that you select. By using these methods, you can generate the following types of reports:Current reports: Pie charts that show the count of work items according to the filter criteria that are specified in the work item query. Trend reports: Line charts that show the distribution of work items over the past six weeks according to the filter criteria that are specified in the work item query. After the reports are generated, you can easily change the date range.Each report includes several worksheets, and each worksheet shows a PivotTable report and a PivotChart report that derives data from the SQL Server Analysis Services cube.
4. Team members can use the Bugs dashboard to determine whether they are managing the list of active Bugs according to established team goals and agile practices. By unit testing each increment of code before check-in, the team can reduce the overall number of bugs that the team must find. A team that focuses on being able to ship each increment of code removes defects incrementally and minimizes ongoing bugs.By using the Bugs dashboard, the team can answer the following questions:Is the number of active Bugs acceptable based on team goals? Is the team postponing too many Bugs?Is the team finding, fixing, and closing Bugs quickly enough to meet expectations and at a rate that matches previous development cycles? Is the team addressing high priority bugs before lower priority bugs?Does any team member need help in resolving bugs?
5. You can use the Code Coverage and Code Churn reports to answer the questions that are listed in the following table. Which builds succeeded? Which builds have a significant number of changes to the code? How often are builds succeeding?How volatile is the code base?How much of the code is the team testing?How high is the quality of the builds? Is the quality increasing, decreasing, or staying constant?
6. After the team has started to find and fix bugs, you can track the team's progress toward resolving and closing bugs by viewing the Bug Status report. This report shows the cumulative bug count based on the bug state, priority, and severity.
7. You can use the Bug Trends report to help track the rate at which your team is discovering and resolving bugs. This report shows a rolling or moving average of bugs being reported, resolved, and closed over time. When you manage a large team or a large number of bugs, you can monitor the Bug Trends report weekly to gain insight into how well the team is finding, resolving, and closing bugs.The Bug Trends report calculates a rolling average of the number of bugs that the team has opened, resolved, and closed based on the filters that you specify. The rolling average is based on the seven days before the date for which it is calculated. That is, the report averages the number of bugs in each state for each of the seven days before the date, and then the result is divided by seven.
8. As the team resolves and closes bugs, you can use the Reactivations report to determine how effectively the team is fixing bugs. Reactivations generally refer to bugs that have been resolved or closed prematurely and then reopened. The reactivation rate is also referred to as the fault feedback ratio.You can use the Reactivations report to show either bugs or user stories that have been reactivated. As a product owner, you might want to discuss acceptable rates of reactivation with the team. A low rate of reactivations (for example, less than 5%) might be acceptable depending on your team's goals. However, a high or increasing rate of reactivations indicates that the team might need to diagnose and fix systemic issues. The Reactivations report shows an area graph of the number of bugs or stories that are in a resolved state or that have been reactivated from the closed state.
9. The Build Quality Indicators report shows test coverage, code churn, and bug counts for a specified build definition. You can use this report to help determine how close portions of the code are to release quality. Ideally, test rates, bugs, and code churn would all produce the same picture, but they often do not. When you find a discrepancy, you can use the Bug Quality Indicators report to examine the details of a specific build and data series. Because this report combines test results, code coverage from testing, code churn, and bugs, you can view many perspectives at the same time.
10. The Build Success Over Time report provides a pictorial version of the Build Summary report. The Build Success Over Time report displays the status of the last build for each build category run for each day. You can use this report to help track the quality of the code that the team is checking in. In addition, for any day on which a build ran, you can view the Build Summary for that day.
11. The Build Summary lists builds and provides information about test results, test coverage, code churn, and quality notes for each build.The data that appears in the Build Summary report is derived from the data warehouse. The report presents a visual display of the percentage of tests that are passing, code that is being tested, and changes in code across several builds. You can review the results for both manual and automatic builds, in addition to the most recent builds and continuous or frequent builds. The report lists the most recent builds first and contains build results that were captured during the specified time interval for all builds that were run, subject to the filters that you specified for the report.At a glance, you can determine the success or failure of several build definitions for the time period under review.
12. After a team has worked on one or more iterations, also known as sprints, you can determine the rate of team progress by reviewing the Burndown and Burn Rate report. Burndown shows the trend of completed and remaining work over a specified time period. Burn rate provides calculations of the completed and required rate of work based on the specified time period. In addition, a chart shows the amount of completed and remaining work that is assigned to team members. You can view the Burndown and Burn Rate report based on hours worked or number of work items that have been resolved and closed.
13. After the team has estimated its tasks and begun work, you can use the Remaining Work report to track the team's progress and identify any problems in the flow of work. The Remaining Work report summarizes the data that was captured during the specified time interval for each task, user story, or bug based on the filter criteria that were specified for the report. The data is derived from the data warehouse.You can view this report in either the Hours of Work view or the Number of Work Items view. The first view displays the total number of hours of work for the specified time period and the team's progress toward completing that work. The second view displays the number of work items for the specified time period and the number of work items in each state. Each view provides an area graph that charts the progress of completed work against the total estimated work for the specified time duration.
14. After work has progressed on several iterations, also known as sprints, you can view the team progress by viewing the Status on All Iterations report. This report helps you track the team's performance over successive iterations. For each iteration that is defined for the product areas that you specify, this report displays the following information: Stories Closed: The number of user stories that have been closed. These values are derived from the current values specified for the iteration and the state of each user story.Progress (Hours): A two-bar numeric and visual representation that represents the values for Original Estimate (grey), Completed (green) and Remaining (light blue) based on the rollup of hours that are defined for all tasks. These values are derived from the current values that are specified for the iteration and the hours for each task. Bugs: A numeric value and visual representation for all bugs, grouped by their current states of Active (blue), Resolved (gold) and Closed (green). These values are derived from the current values that are specified for the iteration and the state of each bug.
15. The Stories Overview report lists all user stories, filtered by area and iteration and in order of importance.Work Progress% Hours Completed: A numeric value and visual representation that shows the percentage of completed work based on the rollup of baseline and completed hours for all tasks that are linked to the user story or its child stories.Hours Remaining: A numeric value for the rollup of all remaining hours for all tasks that are linked to the user story or its child stories.Test StatusTest Points: A numeric value that represents the number of pairings of test cases with test configurations in a specific test suite. For more information about test points, see Reporting on Testing Progress for Test Plans.Test Results: A numeric value and visual representation that shows the percentage of test cases, grouped according to the status of their most recent test run, where the options are Passed (green), Failed (red), or Not Run (black).Bugs: A numeric value and visual representation that shows the number of bugs that are linked to the test case or user story, where the options are Active (blue) and Resolved (gold). If a user story is linked to one or more child stories, the values represent a rollup of all bugs for the user story and its child stories.User Stories that Appear in the ReportThe Stories Overview report lists and highlights user stories according to the following criteria:Stories appear in order of their importance, based on their assigned ranking.Stories appear in bold type when they are in the active or resolved state. Stories appear in normal type when they are in the closed state. Stories appear in gray type when their assigned iteration or area is outside the filtered set, but they have tasks or child stories that are within the filtered set of iterations or product areas.
16. The Stories Progress report lists all user stories, filtered by product area and iteration in order of importance.This report displays the following information for each user story that appears in the report: Progress (% Completed): Numeric value that represents the percentage of completed work based on the rollup of baseline and completed hours for all tasks that are linked to the user story or its child stories.Hours Completed: A visual representation of the completed hours, displayed as a dark green bar.Recently Completed: A visual representation of those hours completed within the time interval specified for Recent (Calendar) Days, displayed as a light green bar.Hours Remaining: Rollup of all remaining hours for all tasks that are linked to the user story or its child stories.The Stories Progress report lists and highlights user stories according to the following criteria:Stories appear in order of their importance, based on their assigned ranking.Stories appear in bold type when they are in the active or resolved state. Stories appear in normal type when they are in the closed state. Stories appear in gray type when their assigned iteration or area is outside the filtered set but they have tasks or child stories that are within the filtered set of iterations or product areas.
17. The Requirements Progress report shows the status of completion as determined by the tasks that have been defined to implement the requirement.
18. The Requirements Overview report presents a snapshot of the work that has been performed for the filtered set of requirements to the current date.
19. By reviewing a release burndown report, you can understand how quickly your team has delivered backlog items and track how much work the team must still perform to complete a product release. A release burndown graph shows how much work remained at the start of each sprint in a release. The source of the raw data is your product backlog. Each sprint appears along the horizontal axis, and the vertical axis measures the effort that remained when each sprint started. The amount of estimated effort on the vertical axis is in whatever unit that your scrum team has decided to use (for example, story points or hours).
20. By reviewing a sprint burndown report, you can track how much work remains in a sprint backlog, understand how quickly your team has completed tasks, and predict when your team will achieve the goal or goals of the sprint.A sprint burndown report shows how much work remained at the end of specified intervals during a sprint. The source of the raw data is the sprint backlog. The horizontal axis shows days in a sprint, and the vertical axis measures the amount of work that remains to complete the tasks in the sprint. The work that remains is shown in hours. A sprint burndown graph displays the following pieces of data: The Ideal Trend line indicates an ideal situation in which the team burns down all of the effort that remains at a constant rate by the end of the sprint. The In Progress series shows how many hours remain for tasks that are marked as In Progress in a sprint. The To Do series shows how many hours remain for tasks that are marked as To Do in a sprint. Both the In Progress and the To Do series are drawn based on the actual progress of your team as it completes tasks.
21. Toward the end of an iteration, you can use the Unplanned Work report to determine how much work was added to the iteration that was not planned at the start of the iteration. You can view the unplanned work as measured by work items added, such as tasks, test cases, user stories, and bugs. Having unplanned work may be acceptable, especially if the team has scheduled a sufficient buffer for handling the load of unplanned work (for example, bugs). On the other hand, the unplanned work may represent a real problem if the team does not have the capacity to meet it and is forced to cut back on the planned work.The Unplanned Work report is useful when the team plans an iteration by identifying all work items that they intend to resolve or close during the course of the iteration. The work items that are assigned to the iteration by the plan completion date of the report are considered planned work. All work items that are added to the iteration after that date are identified as unplanned work.
22. The Test Case Readiness report provides an area graph that shows how many test cases are in the Design or Ready state over the time period that you specify. By reviewing this data, you can easily determine how quickly the team is designing test cases and making them ready for testing. When you create a test case, it is automatically set to the design state. After the team has reviewed and approved the test case, then a team member should change its state to Ready, which indicates that the test case is ready to be run.
23. The data that appears in the Test Plan Progress report is derived from the data warehouse and the test results that are generated when tests are run by using Microsoft Test Manager. The report presents an area graph that shows the most recent result of running any test in the specified test plans over time. For more information, see Running Tests.The horizontal axis shows days in a sprint or iteration, and the vertical axis shows test points. A test point is a pairing of a test case with a test configuration in a specific test suite. For more information about test points, see Reporting on Testing Progress for Test Plans.