Pluralsight Flow Review – Is It Worth It

Managing a team is a time-consuming process. You have to manage their productivity, internal and external relationships, expectations from upper management, and so much more. Any tool that you can use to decrease your workload as a team leader is a lifesaver, allowing you to focus your attention on other more urgent matters. There’s nothing worse than realizing after two or three months that your team has been working inefficiently, and then trying to make up that lost time as deadlines begin looming.

Pluralsight Flow is a new product from Pluralsight that is looking to take some of that administrative burden from you. Flow was designed to give you access to a dashboard that will allow you to view a variety of metrics regarding your team’s productivity and practices. The data that Flow collects would take an extraordinary amount of time for you to collect yourself and promises to give you unprecedented insight into the inner workings of your team.

In this article, we’ll discuss the various features of Pluralsight’s Flow, how it can be used to help your organization in day to day operations, and whether it’s worth the price. If you’d like to know more about Pluralsight itself, we previously posted a review article that will outline everything you need to know.

Pluralsight Flow – Code Metrics

To begin to get an idea as to how Pluralsight Flow can help you more effectively manage your team, we’ll first cover the metrics that Flow will use to give you an overview of your team. The following metrics serve to provide you an idea as to how efficiently code is being added to your project.

  • Coding days
  • Churn
  • Commits per coding day
  • Efficiency
  • Help others
  • Impact
  • New work
  • Productive throughput
  • Raw throughput
  • Risk
  • tt100 Productive

Coding Days

Pluralsight Flow will track the coding days of each of your team members. A Coding Day is counted any day during which your team member pushes a commit to the project on the Git system. This metric allows you to see how often each of your team members commits new code, and as a result, gives you a decent idea as to which of your team members is most productive.

Flow is designed to give you an unprecedented look into the dynamics of your team. A low number of coding days does not necessarily mean that a team member is not carrying their weight. You’re going to want to use this metric, alongside other metrics, and the assigned workflows of those team members to determine the meaning behind this statistic.

Churn

One of the more interesting statistics tracked by Pluralsight Flow is the Churn of the code produced by each of your team members. Pluralsight Flow will keep track of the commits that are made and monitor to determine if these commits are modified by the same team member within the next three weeks. Some churn should always be expected with any team, however, if you are seeing a large amount of churn with any of your team members, it may be worth some investigation.

Commits Per Coding Day

Alone, this metric can only tell you the development style of the individual who is writing the code. You’ll be able to discern whether they are contributing all their code in one push, or if they are pushing multiple smaller solutions throughout their workday. In projects where more than one engineer is working on the same or similar solution, you are going to want to ensure that your engineers are committing smaller amounts of code more frequently, to ensure that everyone is using the same codebase for development.

Efficiency

Following closely in the footsteps of Churn, Efficiency is a measure of the amount of code submitted that is productive. Productive code is code that has not been rewritten and resubmitted by the engineer. Being able to see how often an engineer is resubmitting only slightly modified work will give you a good idea as to the effectiveness of each of their commits.

Late in a project, you may begin to see efficiency drop as the codebase is largely complete and there are only rewrites and the occasional new section of code added. Throughout a project’s life cycle, you will likely see varying amounts of efficiency provided by your team. It’s important to take into consideration the stage of development that you are at, as well as the other metrics available when considering each of your team member’s efficiencies.

Help Others

This metric is very similar to Churn; however, this will measure how often one of your engineers will modify code that has been submitted by another team member within the last three weeks. An interesting statistic, the source of which can be any number of different scenarios. You would most likely expect to see your senior engineers with a high Help Others statistics, and that would likely be less concerning than a junior engineer with a high help others metric.

A high help others statistic in and of itself does not necessarily indicate that the changes being made are for the best interest of the project. It is important to note that this statistic provides more information about how well your team is working together than it does about the quality of the code written either before or after the change. One could reason that if a senior engineer modifies a junior engineer’s code, it could likely have been to correct an error. This does not always mean that is the case, however, and rather than simply committing over the other engineer’s code, it may have been more productive for the two team members to sit together and discuss the reasons for changing the codebase.

Impact

The Impact measure is a representation of how extensive the edits of your team members are to the codebase itself. When your team pushes new code to the base, the size of the push can vary from one line of code changing one tiny detail to a complete rewrite of that file. Using this metric, you’ll be able to ascertain the types of edits that each of your employees is making and identify trends in the types of projects they are taking on.

New Work

A very useful metric, New Work will let you know how many commits your engineers have made for code that doesn’t replace or update any pre-existing code. Though the occurrence of new work is likely to dry up as a project progresses, for the first few months of a new project, or when your team is in the process of adding new features, this metric can tell you how much each of your employees is contributing to the build.

Productive Throughput

Productive Throughput describes any code that survives the Churn or Helps Other periods. This is code that is pushed to the repository and then remains there after three weeks. Ideally, all code that is written will survive this length of time, but in practice, it is very common for code to be written, and rewritten as conflicts with other aspects of the program, or new features are added. As a manager, it is useful for you to see how much each of your employees is contributing to the overall project, and how much of that work subsequently survives and makes it into the final build.

Raw Throughput

A very unspecific metric, Raw Throughput is the sum of all code contributions made by your team member, regardless of what workflow that code is part of, or what happens to that code after it has been pushed. Useful for determining the overall productivity of your workers, this data can be misleading if a lot of the code is subsequently rewritten or deleted.

Risk

An interesting metric, the Risk is a measure of the effort put forth to resolve a bug found in the codebase. Sub metrics such as the size of the commit, how many different files have to be modified, the number of lines affected, and more are taken into account to identify portions of the codebase which should be reviewed, and have a higher probability of containing an issue.

tt100 Productive

Heading into the territory of determining how much a team member is contributing, the tt100 Productive metric is an average of the time it takes for each of your team members to contribute one hundred lines of code after correcting for churn. As this metric removes the time that the engineer spends rewriting or refactoring their code, this measure will serve to provide you with a fairly good indication as to how much each engineer has contributed to the project.

back to menu ↑

Pluralsight Flow – Review Metrics

Review is an important aspect of project development. These metrics will give you an overview of how efficiently your review process is working and allow you to determine in which areas improvement could be made.

  • Reaction time
  • Involvement
  • Influence
  • Review coverage

Reaction Time

Reaction Time is the average time it takes for a reviewer to respond to a comment, or to review a pull request. For the sake of productivity, it is in the project’s best interest that your reviewers can respond as quickly as possible. Ensuring that the review process occurs promptly is essential to an effective development process.

Involvement

A measure of the overall pull requests that each reviewer participated in, the Involvement metric gives you valuable insight into your team’s operations. This is very useful in determining who is involved in most of the review for your project and may help to determine if additional reviewers are required as well.

Influence

The Influence is a measure of how often changes are made to the code submitted after a reviewer has left their comments. As the review process is an excellent teaching opportunity, having code changes made after the fact can give you an idea as to how often the reviewer can identify a potential problem, and then work with the engineer to find a suitable solution.

Review Coverage

Review Coverage is a measure of the amount of code that is reviewed in each pull request. This will serve to give you a better idea as to how much of the code that is submitted to your project has been reviewed before being pushed to the master branch. Using this information, you may identify a need for more code to be reviewed, or perhaps if the influence of your reviewers is quite low you may realize that fewer reviewers are required. This is most likely to occur on a team of very experienced developers.

back to menu ↑

Pluralsight Flow – Team Collaboration Metrics

Ensuring that your team is working together efficiently is a large part of why Pluralsight Flow was created. With these metrics, Flow will reveal to you the details of how well your team is working together, and where you can make improvements for your team if necessary.

  • Time to resolve
  • Time to first comment
  • Follow-on commits
  • PR activity level
  • Ticket activity level

Time to Resolve

The average time it takes to close a pull request is stored in the Time to Resolve metric. Though this may be largely dependent on the size of the request, it should provide you with an excellent idea as to how long it takes for a new piece of code to clear the review process and be ready to merge with the master branch.

Time to First Comment

This metric keeps track of how long it takes from the time they pull request is created to the point where the of the reviewer comments are added. Ideally, your team is not kept waiting for too long before receiving feedback during a pull request, and the lower your team can keep this number the better.

Follow-on Commits

Tracking the number of commits that are added onto a pull request after the request is opened, Follow-on Commits will help you to visualize how your teams pull requests move through the review process. While it is completely unrealistic to expect there to be no commits after a pull request has been created, if you’re finding an abnormally large number of these commits, it may be worth investigating whether your team’s deadlines are too strict, or if some other factors could be manipulated to better serve the team.

PR Activity Level

Less to do with your team overall, but rather to do with each pull request, this metric will allow you to identify pull requests that have attracted a flurry of activity from your team. These pull requests are likely to be of high importance to your team, and where a lot of your team’s attention is focused.

Ticket Activity Level

Keeping your project on track for delivery is a constant struggle, and the Ticket Activity level metric is another metric that will help you in this process. Much the same as the PR Activity Level, this applies to the tickets themselves rather than to your engineers. You’ll be able to identify any problem areas in your current build, and potentially move additional engineers onto a particularly difficult problem, keeping your project on time.

back to menu ↑

Pluralsight Flow – Knowledge Sharing Metrics

Understanding how knowledge moves and is stored throughout your team is a complicated process. For some activities, it’s as simple as knowing that one of your team members is the expert, but other times it’s not so clear. Using these metrics, you’ll be better able to identify how knowledge is transferred throughout your team, and whether any modifications need to be made to your team to better facilitate the development of your project.

  • Sharing index
  • Number of PRs reviewed
  • Number of users reviewed
  • Requirements and security

Sharing Index

The Sharing Index is a generalized measure of how information is flowing through your team. By analyzing who is reviewing and commenting on pull requests, you’ll be able to identify individuals through which all activity is traveling through, or whether the review of code is done uniformly by your engineers.

Number of PRs Reviewed

A very basic statistic that simply provides the number of pull requests that have been reviewed.

Number of Users Reviewed

This metric will help you to see the number of users that submitted pull requests that are subsequently subjected to review. This will give you an idea as to how the review is spread throughout your team.

back to menu ↑

Requirements and Security

With the sheer amount of data that Pluralsight Flow relies on to be able to provide these metrics, it’s reasonable to question what the requirements are for your team to use Pluralsight. Pluralsight was designed to work with a variety of Git Providers, and even if you maintain your own Git on your servers. Pluralsight can access your repositories using SSH key pairs, and then the processing of your data is completed on Pluralsight servers.

For enterprise customers, Pluralsight offers an on-site version which can be installed 100% on your servers. This version would not contact Pluralsight servers and would not transmit any of your information to Pluralsight. It’s important to note however that even for the regular version, the data is not stored by Pluralsight, and no copy is created. Their servers are used to properly analyze the data before returning it to you in it’s more functional form.

Pluralsight Flow currently works best in the newest version of Chrome, though the Pluralsight team is working to bring browser compatibility to more browsers as time goes on.

back to menu ↑

Pricing

Standard – STARTING AT $499 active contributor / year

  • Deep visibility and snapshots into your software development process
  • 50 Repos
  • 30-day Free Trial
  • GitHub, Bitbucket, GitLab
  • Operational Insights
  • Code Fundamentals
  • Email & Phone Support
  • Single Sign-on available
  • API Access available

Plus – STARTING AT $599 active contributor / year

  • STANDARD FEATURES PLUS
  • Maximize productivity and accelerate your product release cycles.
  • Unlimited Repos
  • 30-day Free Trial
  • Jira, GitHub Issues, Rally
  • Code Review & Collaboration Insights
  • Advanced Pull Request Reports
  • Email & Phone Support
  • Single Sign-on available
  • API Access available

Enterprise – Need to contact Pluralsight team

  • ALL FEATURES PLUS
  • Custom plans for larger teams with enterprise requirements
  • Unlimited Repos
  • Flexible POCs Available
  • GitHub Enterprise, Bitbucket Server, GitLab Enterprise
  • Self-hosted or cloud-hosted
  • Priority Support and SLAs
  • Get our highest level of support
  • Single Sign-on available
  • API Access available

Pluralsight Flow Pricing
Pluralsight Flow Pricing
back to menu ↑

Verdict

Managing a team is a difficult task, made more difficult when there are many moving pieces in the project that your team is constructing. The metrics we covered above can be used to provide you the complete picture of how your team is operating and will help to fill any gaps that you aren’t able to directly observe. You’ll be able to be more involved in each of your team members’ duties and use this newfound information to help your team grow and achieve its goals.

The way Pluralsight Flow was designed, it’s not possible to game the system, the data would still be representative of how your engineers are completing their tasks. Whether your engineers commit in one big section, or many smaller sections throughout the week, using the variety of metrics available, their overall tt100 Productive would remain the same. Further, even if an employee just resubmitted the same section repeatedly trying to appear more productive than they were, the system would identify that as churn, and it would not be included in the tt100 Productive.

From the standpoint of ensuring that your team is as efficient as it can be, and to identify weaknesses in your team and help you come up with a plan to solve them, I would recommend Pluralsight Flow for all of the benefits it can bring to you and your team. Pluralsight Flow will help to make you and your team better and is an excellent addition to any manager’s arsenal.

      IQ Unlock