Getting started

In this post, I'd like to share the use cases of story points during LINE Pay's app and server development. I hope this post will provide you with an opportunity to understand story points better and make good use of them.

What are story points?

Before directly diving into story points, let's first take a look at three concepts related to story points.

• User story: Describing a feature or functionality of a software system, so-called "requirements" from an end-user perspective. Not all requirements constitute as a user story so it is important to pick those that are meaningful to users. Let's say we're going to implement a sign-up feature. Normally we'd put it down as "we need to provide a sign-up feature for users to use the LINE Pay service." Instead, this can be a user story when it is written from a user's perspective: "Non-LINE Pay users will agree to the terms and conditions and click the sign-up button to become a LINE Pay member in order to use various features offered by LINE Pay." Often user stories follow a template such as "As a <user type>, in order to <objective>, I will <use feature>." However, this is not a must. What's most important for user stories is putting yourself in users' shoes and think like them.
• Sprint: A time-boxed iteration of a continuous project. Say each sprint is 2 weeks long. It means that a set amount of work is planned, completed and reviewed at the end of every 2 weeks.
• Velocity: Metric to estimate how many story points can be earned for each sprint. Story points are earned when a user story is implemented and delivered as a functioning feature.

Concept

Story points indicate how difficult it is to implement a user story into a working feature. This value is an objective indicator as story points reflect the perception of many in a given group instead of relying on an individual user. We can compare it with MD (man-day), which is a more familiar concept for us. MD is one of many productivity metrics, quantifying how long it takes for a single developer to implement a specific feature. For example, we'd say it takes 1 MD if a developer called Brown spends a full working day to develop a sign-up feature. MD quantifies efforts (or productivity) using two barometers, namely person (in our case, Brown) and time.

Unlike MD, story points do not use a person and time as barometers. It is entirely focused on user stories for implementation. There has to be a baseline criteria to measure the difficulty of implementing a specific user story, and story points represent relative difficulty against the criteria. Let's assume we have user stories for sign-up, log-in and log-out. Among those three user stories, log-out user story is the easiest one to implement. Use the log-out user story as a baseline and estimate relative difficulty of implementing sign-up and log-in user stories against the log-out user story as a baseline.

Let me give you another example to help you better understand. It is about 325.5 km straight from Seoul to Busan. It will naturally come to you that it is a very long distance because we already know how far "1 km" is. Since we know how long it takes to travel 1 km, we can easily guess 325.5 km is far, far away. The same goes for story points. We select a baseline just like 1 km in the distance example and set its story points. Then, we compare it with respective user stories to quantify the degree of difficulty for implementing those user stories. We'll go back to the distance example again and compare MD and story points. Travel time from Seoul to Busan varies, depending on the type of transportation. It takes about 5 hours by car, 2 hours and 40 minutes by train and 1 hour by plane. Travel time is different per transportation, but the distance from Seoul to Busan is still 325.5 km. This example clearly shows the difference between MD and story points. MD is the travel time per transportation while story points are the distance that is constant regardless of transportation. In other words, each developer would require different MDs to implement a user story, but story points do not change as they entirely rely on user story itself.

Benefits

You can benefit from story points in a number of ways. It is a quick and easy way to understand the current status of your team and estimate and plan for the team. In addition, you can continue to rely on more objective metrics as story points are less affected by external factors. Simply put, when you rely on story points to estimate and plan, you can be prepared for those many meetings during a sprint or all exceptions such as a system failure or sick leaves.

As mentioned above, MD uses person and time as a barometer. While Brown needs 1 MD to implement the sign-up user story, Cony could require 3 MDs for the same job as she is not familiar with the existing system and programming language. This shows that the same user story would require different MDs, depending on who is working on it. Such inconsistency makes it difficult to objectively measure a project size, and estimation and planning become inaccurate as scheduling depends on individual capabilities. Unlike MD, story points are kept constant regardless of who is implementing a user story. In other words, when the sign-up user story is assigned 3 story points, this will remain as 3 story points whether it is implemented by Brown or Cony. Such characteristics of user story and story points allow you to plan strategically.

How to use story points

Now that we have grasped the concept of story points, we can move on to how to use story points.

Choose a baseline user story

Story point estimation is a kind of relative estimation, comparing one user story against another. Therefore, we need to define a user story to be used as a baseline like "1 km" in the distance example above. (This baseline user story is also called "golden story.") Before selecting a golden story, however, it is recommended to define the range of story points first. Simply you can use fingers to represent the range of 1 to 5, t-shirt sizes of XS, S, M, L and XL or a sequence like the Fibonacci sequence (0, 1, 2, 3, 5, 8, …). You can pick what suits your team the best. For this post, we used the Fibonacci sequence adopted by the LINE Pay team.

It is a good idea to set a maximum when using a numerical sequence like the Fibonacci numbers. We've set the maximum at 13 for this blog post, which gives us 0, 1, 2, 3, 5, 8, and 13 as a candidate for story points. Once you've set the range, you need to define the baseline. You can choose a median like 3 as a baseline or the minimum such as 1 to assign an incremental value to respective story points. For example, "0" can be assigned to a user story that is meaningful for end-users and can be implemented with very little effort. For illustration purpose, we will use 1 as the baseline here. We've previously used sign-up, log-in, and log-out user stories. Say the log-out user story is the least difficult among the three user stories, and this will be used as golden story. Then, we estimate story points for sign-up and log-in in reference to log-out. In addition, when the log-out user story is used as a baseline, there should be no other user story which is easier than the log-out one. If there is one, it must be easy enough to get the story point value of 0.

Estimate story points

Story point estimation is the most important step of all. As the term, estimation, indicates, we don't require an accurate value for story points. Many try to estimate as accurate as possible during this step. However, this is a bad example of how to use story points. Furthermore, it goes against the purpose of Agile, let alone story points. You could start having a doubt at this point, "Isn't it still too abstract although it doesn't have to be accurate." Dear readers, have no worries. We'll start with a rough estimate but it will become more accurate with continuous repetition.

The law of large numbers (LLN) explains why I say it will get closer to an accurate value. If you want to refresh your memory about the LLN or quickly learn about it, watch the section on the LLN in this documentary called "Nova - Prediction by the Numbers".

Many find it hard to do estimation in the beginning and end up substituting the number of hours to estimate story points. For instance, they automatically assign 1 story point for a job that takes 4 hours to complete and 2 for 8 hours and so on. Then, they stop using story points, thinking that it is not much different from previous estimation methodologies. This is a very wrong approach. If this sounds familiar to you, please read this post and shift your perspectives on story points.

You can either set aside some time to conduct story point estimation or do it during the planning stage when sprints start. One of the common practices for story point estimation is "planning poker." This is how it's done: Pick a user story and briefly explain it. Each member mentally formulates an estimate. Then everyone holds up a card with the number that reflects their estimate. Assign the estimate that wins the most votes. If opinions are divided, take just a couple minutes to understand the rationale behind different estimates and do another round of poker (Refer to the WIkipedia article on planning poker for more information).

Remember that the estimation process should be quick and easy. Think intuitively and quickly, "How much more is B difficult compared to A?" For example, if you think the log-in user story is twice as difficult as the log-out user story, the story point for the log-in user story should be 2. Let me give you another example with the sign-up user story.

• Brown: I think it is about 3-fold more difficult than the log-out user story.
• Cony: I think it is about 5-fold more difficult than the log-out user story. There are many things to think about.
• Leonard: Well, I think it is more than 3-fold but not up to 5-fold.

Under this scenario, the story point for the sign-up user story will be "5." Story point estimation does not require ultimate accuracy. And, it is better to choose "5" over "3" when there is someone like Leonard who is uncertain.

Apply to your project

So far, I haven't mentioned the most important factor. Story point is a measure commonly accepted by a team, working on the same project. When Team A and B accomplish 20 and 30 story points per sprint respectively, can you say Team B is more productive compared to Team A? The answer is that "story points shouldn't be used as a basis for comparison." Since Team A and B have different baseline, it cannot be an apple-to-apple comparison. Comparison based on story points is same as comparing one unit of distance between Country A and B when Country A uses metric units (for example, km for distance) and Country B uses customary units (for example, mile for distance).

If you've prepared up to this point, it is time to strategically establish a plan based on prioritization with a quadrant analysis or Kano model. Both methods help to choose which one to prioritize based on the amount of value to users and story points.

We'll now take a look at how to use story points with a few examples. Story points are often used in forecasts and planning. Think of a team of five members using story points for the first time with a 2-week sprint. They have completed story point estimation for user stories to be implemented. Since it is a first try for the team, the team simply makes an assumption that one member can achieve 3 story points per sprint, and it makes a total of 15 story points per sprint for the entire team. Now the team takes out user stories worth 15 story points from the backlog including sign-up for 5 story points, log-in for 3 story points and log-out for 1 story point. Once they have 15 story points on the plate, they make a plan for the first sprint and embark on the job.

After 2 weeks when the first sprint is completed, the team checks how many story points they have managed to complete. If they manage to achieve 10 story points, they could adjust their plan for the second sprint, targeting 13 story points instead of 15 story points from the previous sprint. After the second sprint is completed, the team checks again how many story points they have managed to complete. For your information, if one of your team members completes about a half of the given user stories, it does not mean a half of the given story points is completed. Story points are earned only when the given user story is implemented and delivered during the given sprint. Usually after about three sprints, the team can more or less accurately determine the maximum story points per sprint for the team. This is called velocity. When a team assesses it can achieve 13 story points per sprint, the team's velocity is 13 story points, and it can plan, estimate, and conduct a retrospective based on this velocity. Velocity can be used in the estimation process as follows. It is very straight forward. When a specific epic (for example, a bundle of user stories) has 60 story points and the team's velocity is 13 story points, this team requires about 10 weeks to complete this epic.

During a retrospective meeting, the team evaluates the finished sprint based on completed story points and velocity at the team level. If the team earned more story points than its velocity, they should look into how it was possible. They should also discuss whether it is necessary to raise the team's velocity. On the other hand, if the team earned less story points than its velocity, they should review to find out why it was so. They also need to talk about if they should lower the team's velocity. While doing so, the team defines action items to eliminate obstacles and share best practices to gradually increase its velocity and become a more efficient team.

There is another important message to deliver at this point. There is no need to blame oneself even if they fail to fully earn story points assigned during the planning stage. Unexpected surprises always occur as we work on our mission such as a series of meetings or system failures. Velocity of a team should encompass all of those factors. Say your team could not earn story points in the previous sprint because of a few unexpected situations but do not expect that would be the case in the next sprint, your team can still plan for story points higher than its velocity for the upcoming sprint. The success of a sprint is not determined by whether your team fully earn the story points planned for that sprint. More accurately, story points should be considered as an indicator of the status of the given sprint. If your team's velocity consistently drops, then you should thoroughly investigate and eliminate those external factors, affecting your team's velocity.

Use case in LINE Pay

Let's take a look at how LINE Pay is using story points. I might have explained key concepts too briefly above. I'm afraid such a short summary could be misleading or cause readers to misunderstand the essence of story points. I'll try to elaborate further with LINE Pay's case. I started out with LINE Pay server development, built and worked at LINE Pay App Development Team for almost 3 years and now back to Server Development Team. Let me compare cases from each team to elaborate on how to use story points.

Write user stories

First I'll talk about how we created user stories for LINE Pay.

Define users

There are two types of users for LINE Pay: customers who use the LINE app to make a payment and merchants who provide a service through LINE Pay. Who should the LINE Pay Team consider as its users, customers or merchants? Well, it should be noted that they are not the only users for LINE Pay, and the definition of users vary per team. When I was working at the App Development Team, most user stories were defined for customers or merchants, depending on whether it is for the LINE Pay app or merchant app. However, it was a different story for the Server Development Team. Some user stories were indeed for customers or merchants just like in the App Development Team, but many were for different types of users such as app developers or business team members. This clearly shows that user stories are not written only for end-users. Anyone who is in contact and uses the value created by the team should be defined as a user. For example, when the Server Development Team defines a user story for implementing APIs so that the App Development Team can use those APIs, the App Development Team is the user in this case.

User story format

I recommend not to get fixated on the format when writing a user story. Don't waste your energy on writing user stories with phrases like "As ~, I will ~" or "In order to ~, as a ~, I want ~." Instead, you should focus on delivering real value to users with your user story. We thought from users' perspective and assigned story points only when user stories would offer value to users. Otherwise, user stories were given 0 story points. In addition, we broke user stories down into the smallest piece of work possible. For instance, instead of "LINE Pay users should be able to log in and out," we defined this as two user stories: log-in and log-out. We remained truthful to "Independent" and "small" from the INVEST mnemonic, which is a reminder for effectively writing user stories. 

Choose a baseline user story

We adopted story points when LINE Pay had already been in service for 4 to 5 years. It was difficult to choose a baseline user story. So we took the following approach.

• Baseline from the App Development Team's perspective: Controlling UI (user interface) on the screen with the given conditions (for example, Display or hide, zoom in or zoom out) without server API communications
• Baseline from Server Development Team's perspective: Providing simple data with the given conditions with a new API

These may appear to be functional specifications from provider's perspective, not user stories. Actual user stories were not written in this way. This was to provide reference as a baseline for story point estimation. In other words, when a user story meets this criterion, it could be considered as 1 story point. Using this approach, we selected a number of baseline user stories and built the following story point reference table to prevent story point inflation.

Estimate and allocate story points

As I mentioned earlier, we only allocated story points to those user stories that delivered real value to users. This is also recommended by many Agile experts. What I mean by "delivering real value to users" is to implement a feature for users to actually use it. The case in point is the log-in user story. When it is implemented and provided, users can log in with this feature. Such a user story is eligible for story points. When story points are allocated only to those that deliver real value to users, you'll witness a small shift in how team members think and approach. This will increase the probability of team members looking for ideas that will deliver real value to users in order to bring up the team's velocity.

Estimate story points

We decided to include a bit of technical background when explaining and discussing about user stories during story point estimation process. We ensured that technical background should be as simple as users can understand and exclude too much technical information such as internal structure. For instance, when the Server Development Team needs to provide 2 APIs to implement a specific user story, it would include the following technical information only.

"Two APIs should be provided. API A delivers information. API B renews information."

We decided not to include technical details as follows.

"API A should combine System C's API and data from a specific database when delivering information."

Use the Fibonacci sequence for story points

LINE Pay uses a sequence from the Fibonacci numbers for story points. This is because the Fibonacci sequence itself encompasses uncertainty. In the Fibonacci sequence, the gap between two numbers becomes wider as the numbers become bigger.

• 1, 2, 3, 5, 8, 13, 21, … : The difference is only "1" from 1 to 3, but it becomes bigger from 3 and onwards.

This indirectly represents that user stories given a bigger story point are likely to face many difficult variables. As for a user story to mark a LINE Pay merchant within a 1.5 km radius from the user's location on the map, for example, how much more difficult would it be compared to the log-out user story? We probably need to consider quite a large number of factors. We are not sure if the location-based search framework is already provided on the LINE Pay platform, and many more questions come to mind. If we were using the arithmetic sequence instead of the Fibonacci sequence, team members would have suggested many different story points from 6 to 10 and more. With the Fibonacci sequence, it is more likely to converge into one of three numbers, namely 5, 8 or 13, and it is easy to grasp that a bigger number means a bigger uncertainty. In addition, those user stories with story points of 13 or bigger should be broken down smaller user stories in order to resolve inherent uncertainty. User story A with 13 story points can be reviewed and broken down into three user stories as follows:

• User story B: 5 story points
• User story C: 5 story points
• User story D: 5 story points

Total story points are 15, bigger than before, but it can be construed that uncertainty has been eliminated by dividing up one big user story.

Assign story points - refactoring and debugging

Should we assign story points for refactoring? How about the following refactoring? "Need refactoring of the passcode related code as it seems too complicated." This is not appropriate neither as a user story nor for story points. To actual users, the value of using the passcode does not change before and after the refactoring. I consider conducting refactoring as an inclusive part of user story or an act of repaying technical debt. Refactoring should be conducted at the same time as implementing a user story. Story points take account of all of these elements. If implementation is focused only on results without consistent improvement, and refactoring is conducted only when there is no other option, it can be considered that story points of this user story have been earned for free without fully completing the task. In this sense, I consider refactoring as repaying technical debt. How about debugging? 0 story point for debugging as well. I say debugging is also one way to repay technical debt. When the log-in user story is delivered but a bug is found, resolving this bug is also repaying technical debt as the log-in user story implementation should have been clear of any bug in the first place. So, no story point is given to debugging.

These two cases lead to a simple conclusion as follows. "Refactoring should be an integral part of the user story implementation so that it doesn't require a separate refactoring step. Also, if implementation is delivered free of bugs, the team would be working on something else to increase user value instead of spending additional time on refactoring or debugging. This approach will eventually improve the team's velocity." There can be a side effect to assigning 0 story point to refactoring or debugging. What if your team is in the following situation? Upon completing a sprint, developer A earned 10 story points while developer B got 0 story point. Looking deeper into details, it turned out that developer A was able to focus on user story implementation without any issue while developer B had to refactor the code of a certain user story implemented by developer C who already resigned. That's why developer B did not earn any story point. In this case, developer B could be demotivated. Developer B finished their work for the sprint but story points indicate otherwise. Therefore, it is important to create a common understanding among team members to avoid unnecessary misunderstanding. Everyone needs to accept that story points are not a measure of individual performance but an indicator of the progress the team is making toward delivering user value.

Assign story points - data extraction

With LINE Pay service, we often get requests from other teams to extract data for strategic planning. Should we assign story points for such data extraction requests? LINE Pay teams decided to assign story points for these requests. Provided that the following rules are complied with.

• Estimate the level of difficulty in case of extraction of a new type of data.
• No story point is given if it is a rerun with a changed period.

We came to this decision because external teams as a user benefit from data extraction we provide. However, when we get the same data extraction request with different dates again, we reference the previous request and grant 0 story point. As we already prepared a means to handle such data extraction with the first request, subsequent requests reuse the existing method. 

Let me give you an example. We delivered a feature to view their payment history during a specific period for LINE Pay users. Users can set a period and check their payment history for the given period. Data extraction requests are similar to this feature in a way as we make a query to perform the job and provide the result upon each request. As we don't earn extra story points every time users use an already implemented feature with different dates, we don't grant story points for data extraction requests with different dates.

There is another tip for data extraction. When there are consistent requests for data with a change in specific conditions, the team's velocity can be improved by providing users with a tool for simple data extraction.

Share sprint data

Sprint data should be easily accessible and shared among team members so that it doesn't create misunderstanding and it can spur on a virtuous cycle. As shown below, LINE Pay also keeps a record of many other items including the number of bugs and user requests and issues without story points handled during a sprint. This helps to check the team's velocity.

You can see from the data above that achieved story points for sprints 1012-1023 and 1026-1106 are less compared to sprint 0914-0925. You can also notice that bugs and inquiries significantly went up for sprints 1012-1023 and 1026-1106. If the team invests time to analyze why there were so many bugs and inquiries during those sprints and makes improvement, they will be able to focus more on earning story points, for example, delivering value to their users.

It should be noted that such stats are for the good of the team and it should not be used to evaluate individual members or compare productivity with other teams. Always keep this in mind.

When those data are compiled, story points achieved by each team member will be available as well. Managers could be tempted to use those data for "other purposes," but they must not. As I've explained before, story points should be used for the team. Story points show where your team stands and how much further it can go. If story points are used as a means to evaluate individual performance, it will not only undermine its inherent value but also lead to a number of negative results such as story point inflation or demotivation of team members.

In addition, story points are defined independently for each team based on their unique characteristics so it won't be an apples-to-apples comparison if you make a comparison between teams based on story points. I've made it clear previously. When one team shows a gradual increase in its velocity and it turns out it was thanks to improved internal process rather than story point inflation, other teams can adopt this process and see if it works for them, too. This is how story points should be used.

Wrap-up

Let me wrap up with a recap of key points I intended to deliver in this post.

• MD and story points are different.
  • Story points indicate the difficulty of a user story itself without concerning who and how long.
• Baseline for story points is discussed and agreed between team members.
  • In other words, story points are a unique unit of value for each team.
• Story points do not require accuracy.
  • It is a way to estimate in a quick and smart way.
• Story points are only assigned when there is a value for users.
• Earning all story points for a given sprint must not be a goal.
  • The goal is to check the current status based on story points earned during a sprint.

Be aware of the following points.

• Do not substitute time to estimate story points.
• Consistently manage to prevent story point inflation.
• Do not use story points as a means to evaluate individuals or compare against other teams.

Mind you, this post does not fully cover everything about how to use story points. If you find it interesting, please research further on your own. There is so much more on the topic of story points. I'm a little worried this blog post could be misleading as I explained it too briefly. LINE Pay's case might not be the only correct answer. However, we strive to uphold the fundamentals of user story and story points. LINE Pay development teams rely on story points as a reality check as to "how our teams work to deliver meaningful value to users" and continue to make improvement. I hope you'll get insight in this regard and find the right direction for your team to adopt and use story points. Sometimes we are trapped in formalities, forget the fundamentals and disparage everything. I'd like to close with the following message to keep us alert.

Throw out the formality and focus on the fundamentals.