The IDP Team’s Ideal Vision: Building Consistent Authentication Solutions Across Mercari Group in the Face of Three Major Issues

In October 2021, Mercari launched the “RFS” project to strengthen the company’s foundation for service and feature development.

“RFS” is an acronym for “Robust Foundation for Speed.” The project was launched to support disruptive growth across the entire Mercari Group, and aims to solve complex technical challenges and drastically strengthen the shared foundation of all Mercari Group companies.

It has now been more than a full year since the launch of the RFS project. So, in this article, we wanted to answer the question that’s on everyone’s mind: “How is it going?” In this series spanning three articles, we will talk to several key people overlooking the three areas of focus for RFS, namely C2C transactions, ID platform, and CS tool, and ask them about the problems faced by the project and how they are addressed. This time, we sat down with Tatsuya Karino (@kokukuma) and Kotaro Oi (@koi), two members of the IDP (ID Platform) Team, to discuss the current state of the project driven by Mercoin’s release and the importance of IDP at Mercari Group.

inding solutions to ensure that everything involving ID is handled consistently across Mercari Group

──First, can you tell us about the IDP Team’s mission and your roles?

@kokukuma：The IDP Team’s mission is to implement appropriate authentication, access control, and data protection for Mercari Group’s services in a standardized form.

As for my role, my official title is Tech Lead, but I think what I actually do is pretty different from what most people envision as a tech lead’s role. In my mind, my main responsibility is to ensure that everything involving ID is handled consistently across Mercari Group. For problems that have a general, clear solution, I encourage people to use that; for problems that don’t, I work to come up with solutions.

When working to come up with solutions, I think about not only my team’s direction, but things other teams are or will be working on. This helps me identify whether or not the IDP Team’s ideal vision for IDP is in line with what Mercari Group is doing. If it’s not, I work to align them again, like by adjusting our vision.

But there’s only so much I can do by myself, so I consider another one of my responsibilities to be ensuring that the IDP Team can do this at a larger scale, rather than just me.

Tatsuya Karino (@kokukuma)

──What about you, @koi?

@koi： My role is Product Manager (PdM), but like @kokukuma, I think my actual responsibilities are just a little bit different from a typical PdM. (laughs) When it comes to coordinating with people outside of the team, @kokukuma already handles most of that, so I work with him to stay up to date with the information we need.

Other than that, I handle things like coordinating schedules for different projects and formulating the team’s roadmap—I try to do as much as I can to avoid putting more work on the engineers. In that sense, I guess my current work is closer to a project manager than a product manager. In particular, I’ve been pretty closely involved with FIDO-related projects recently.

The three main issues faced by the IDP domain

──What would you say are the challenges of RFS in the IDP domain?

@kokukuma：To start, I’d like to clarify that the IDP Team doesn’t actually spend a whole lot of time working on RFS as a project in and of itself. Rather than looking at RFS and other important projects and deciding which is higher priority, we view projects as short-term drivers to help move us closer and closer to our long-term goal. In that sense, all projects we work on are connected to RFS.

There are three main issues in the IDP domain that are aligned with the goals of RFS.

・ The architectural patterns that can be used for authentication and authorization within the company are unclear
・ We don’t have sufficient functions to control privileges held by access tokens
・ Developers can’t configure OAuth clients on their own

I’ll explain each of these in more detail. First, architectural patterns. We don’t have clear patterns of architecture to guide us in how we should use authentication and authorization internally. As a result, when other teams want to create new services using Mercari or Merpay features, we have to make a decision without actually knowing what options or characteristics there are. In this case, we usually talk to people or reference services we know in order to make a decision, but that means any tech debt in the service we reference gets passed down to the new service as well.

The second point is about access token privileges. The IDP Team issues access tokens for external use, but the mechanisms in place to control access privileges haven’t exactly been the best. To be clear, they were sufficient for the use cases at the time, so it’s not like they were bad. But when it came time to extend them and use them in other places, what we had wasn’t enough. That meant that it was difficult for new services to safely use Mercari and Merpay’s IDP. This was a major problem we faced in the RFS project.

The third is about OAuth client configuration. In order to issue tokens, you have to configure an OAuth client. When another team wants to use the API for a new service, they send a request to IDP. The IDP Team had to handle that configuration manually.

Every time that happened, the team had to discuss and decide what to do about the architecture, what cases should be handled how, and so on. But we only have so many resources, and we weren’t able to handle all the requests that we received. From the requesting team’s point of view, that communication itself is a lot of work, and waiting around for the request to go through is a waste of time. So the IDP Team members manually handling this configuration is exactly the kind of thing that RFS is looking to solve.

──It sounds like you were facing some rather complex issues. How much progress has been made since the start of the project?

@kokukuma：Right now, I’d say that we’ve made good progress in resolving the second and third points, but the first is a pretty deep-rooted problem. Like I said earlier, the IDP Team doesn’t spend much time dedicated to RFS; we use other projects as drivers. As we design and plan these projects, we make sure to incorporate the RFS perspective and do things in a way that’s aligned with RFS’s goals.

For the past year or two, the main driver has been the release of Mercoin. Thanks to the various functions and features created in the development process, the second and third issues I listed above are on their way to being resolved. For example, the need for access control for data transmitted between Merpay and Mercoin served as a strong driver to create functions to control access token privileges, which was the second issue. The need to create multiple OAuth clients with complex configurations for Mercoin helped us make progress on the foundation for allowing each team to easily create OAuth clients and see the current state of things, which will solve the third issue. However, since Mercoin came with an entirely new set of business requirements, we had to introduce concepts that didn’t exist within Mercari before, which made the architecture situation even more complex than it already was. As a result, we weren’t able to make progress on solving the first issue.

──I guess it would be too easy if everything could be solved at once, wouldn’t it? But now that two of the three issues are on their way to being resolved, is there anything new you’re working on?

@kokukuma：Right now, we’re working on FIDO and the migration of access tokens used in the client application. These are things that are important in the context of RFS, too. We want to make it so when creating a new service, we don’t have to ask users to install a new app or use some other service, and instead allow them to use the new service within the Mercari app in the same way as our existing features. I think that this will be a requirement for most new services going forward, not just Mercoin.

In developing Mercoin, we were able to implement a process where, when using a service that has a different security level, the user can authenticate using FIDO, obtain a new token with strong privileges, and use that token in Mercoin. This is a big step forward. We want to be able to implement that process for other services, too.

Other than FIDO and access token migration, we hope to be able to steadily support the many projects that other companies are working on. This support will of course have an impact on the business, but it’s also in line with the IDP Team’s ideals for IDP at Mercari. I think that it’s important to not just convey the IDP Team’s own mindset, but also communicate carefully with other teams to ease any concerns about their project. This will help us both update our own ideals and solve the problem of unclear architecture.

──The projects that the IDP Team is involved in are all running on different schedules, right? How do you prioritize your work? I imagine it’s difficult when all projects are high priority to the company working on them.

@kokukuma：It really is. (laughs) While we were working on Mercoin, it was clearly our top priority, so we didn’t have to think about it too much. But now there are plenty of projects that aren’t quite as big as Mercoin, but are very important to individual Mercari Group companies. When everything’s important for different reasons, it’s really difficult to compare them.

As one attempted solution to this problem, the IDP Team decided to make a quarterly roadmap. We’ve only just started, so it’s hard to say whether it really works, but… We create a roadmap that contains all the projects (that we’re aware of) each company is working on for the next two to three years, which gives us a longer-term view than just the upcoming quarter. This roadmap is made available to the rest of the company. Then, if other teams notice that anything is missing, they can reach out to us, and we’ll update the roadmap accordingly.

Working at Mercari as an ID engineer in our current business phase

──What are your goals as a team going forward?

@koi：I’d like to focus on making sure that the rest of Mercari Group understands the IDP Team’s mission and goals, and that other teams feel comfortable reaching out to us if they need anything because they know what we do. We make information about what functions we’re working on open through our roadmap, but if it’s too technical, it can be hard for other people to understand. I feel like we need some sort of “translation” to help people understand what these functions do and what benefits they will have. I’d also like to find ways to make information easier to find for teams looking into new business possibilities.

@kokukuma：That’s a good point. The issue brought up at the start of RFS of architectural patterns being unclear meant that not just the business side, but even engineers didn’t really understand the full story of what IDP functions do and what limitations they have. We weren’t doing a good job of communicating that ourselves, either. I want to fix that. As we do so, I think we’ll also need to make it clear what the use cases for these functions are. For someone like myself who knows a lot about the functions and how they work, it can be pretty difficult to break it down and explain it to other people in a way that’s easy to understand, so I’m hoping a PdM like @koi can handle it better. (laughs)

@koi：We’ll see about that. (laughs) In Merpay, we have a PM All Hands meeting, so I’d like to use that as an opportunity to go through our roadmap and explain what the team is thinking and what functions will be available in the future. Of course, we need to have that communication with Marketplace and Mercoin as well.

We also have a weekly morning discussion that’s open for anyone to join, and I’d like to come up with ways to make better use of it. It’s meant as an opportunity for us to give advice and answer any questions people may have, but it can be a little intimidating for people to ask questions in front of so many engineers, especially if they don’t really know what they should be asking in the first place. I think that’s something us PdMs should work better to accommodate.

Kotaro Oi (@koi)

@kokukuma：On a somewhat related topic, sharing knowledge through blog posts and study sessions certainly isn’t a bad thing, but I think it’s easier for people to learn about topics like this when there’s a reason for them to be interested—like when they actually work with the IDP Team on a project. It would be nice to have materials summarizing use cases and such ready to go so we can share them right away whenever the occasion calls.

It’s hard for both the organization and the business to scale if there aren’t any reference documents or visualizations of how we prioritize projects. As the company gets bigger, I feel like it’s getting more and more important for us to keep our roadmap and documentation updated and improve our communication with other teams.

──We’re just about out of time, but I’d like to ask two last questions. What’s your vision for the future of the team? And, what would you say is the best part about working in the IDP Team now?

@kokukuma：I’d say our vision is the team mission I mentioned at the beginning of this interview: to implement appropriate authentication, access control, and data protection for Mercari Group’s services in a standardized form. The ideal situation would be one where anyone in the company can create a new service and (as long as they create it in a natural way with no technical debt) automatically have appropriate authentication and proper access control to ensure that no personal information is leaked. To do this, we need to improve our communication.

Mercari Group as a whole is seeing the growth of businesses like Merpay and Mercoin that have an integral need for strong authentication. The IDP domain will be very important in management decisions going forward, too. If it weren’t that important to the service, there wouldn’t be much need to spend time on making it versatile, and our work would probably be focused more on short-span projects. But Mercari’s services involve things like payment and cryptoassets, so not taking authentication seriously would be a huge risk to the company and cause us to be unable to continue our business. Thinking about the future of Mercari’s businesses, I think that this is a really valuable and meaningful opportunity for engineers.

Another cool thing about what we do is that we’re able to get an outside view of our work, rather than only looking at the impact inside the company. We joined the FIDO Alliance recently, and we work with external consultants, so we’re able to see from a third-party perspective whether Mercari’s current implementation of ID-related functions is really moving us closer to our vision or not. The experience you can gain by working here will be a very useful asset to your career as an ID engineer going forward.

@koi：In my previous job, when I was working as an ID engineer, I felt that I wanted to place more emphasis on communicating with various teams and people. In that sense, I find it fun to be a PdM and collaborate with stakeholders in other domains, like CS and second-line teams, to move projects forward. This kind of broad communication is a common requirement for teams working on infrastructure, but even compared to other platforms, the ID domain is still fairly small in terms of people, so I think this experience will be a big advantage for me later on.

@kokukuma：One last point I want to emphasize is that Mercari is an awesome place for people that want to discuss ID-related topics, but haven’t had opportunities to do so. Anyone working in this domain has probably asked themselves questions like “Do I really have to use refresh token rotation in all clients…?” or “What’s the best level of token exchange restrictions to use…?” that don’t have clear-cut answers. But not many people are familiar with these topics, so even if you ask other engineers, you often find that they don’t understand the problem in the first place. On top of that, people tend to put off things they don’t understand, so your question gets left unanswered, and whatever the problem was in the first place never gets solved. It’s an unfortunately common situation.

But at Mercari, we have plenty of people who are strongly interested in the ID domain, so you’re actually really likely to find people at your level of knowledge who can—and want to!—discuss questions like these. Plus, it’s pretty clear that the execs have high expectations for us, which is really fun as an engineer. So if you’re passionate about the ID domain and feeling lonely because no one understands the questions and topics you want to discuss, I think you’ll have a great time working at Mercari in the phase we’re in now.