Themes for 2025

DS and Risk themes

To remind everybody, for IPO in 3 years we need to grow by 10x in 3 years. Ingredients for 10x growth in 3 years are

1. Existing customers themseles grow by 2.2x in 3 years - zomato, swiggy, zepto, blinkit, myntra - all are growing
2. Business brings us new customers - myntra, uber, ola etc and lot of D2C merchants. That adds another 2.2x to TPV
3. By improving our systems (approvals etc) we increase TPV by 2.2x in 3 years

2.2x growth in 3 years comes to around 30% growth YoY from DS / risk efforts. Can we increase approval rate, reduce blocking, reduce churn and give more upgrades (from DS/risk efforts - not counting new merchants or merchant growth itself) so as to grow by 30% in one year while keeping delinquecy in place? That is the challenge.

Main projects

1. Improved approvals for PL

   Each successive approval model has led to higher approval rates / lower delinquency. We need to continue that in 2025. Currently the ideas the team has are improving existing batch v6 model and exploring network properties for approvals. Other than that we will need to continue the effort of integrating other sources of data to increase approvals.

2. Approval heuristics for Pi3

   We will work on heuristics to approve additional users on Pi3. In the later half of the year, we may start working on an approval model for Pi3, in case Pi3 grows significantly.

3. Credit line management

   There are two things here. First is that we have have a credit upgrade model - and we should work to improve that. Secondly, since 2024 H2 we have two new capabilities - we have the spend limit feature and we are allowed to reduce credit limit of the users. We need to use these two levers optimally.
   Separately we need to clean up the credit lines. There are lots of legacy lines lying around in the system.

4. User retention system

   There are two aspects here. First is to build better and better models. Second is to ensure that rest of the machninery gets the business gains from the model.

5. Billbox / Select underwriting

   After a long wait, we have finally gotten to write a separate billbox approval model. Billbox is here to stay and we need to keep working on improving approvals and delinquency on billbox.

6. Anti fraud system

   Major improvements that I am looking forward to are: Firstly, improvements in precision of models. Secondly, LTV aware blocking. Other than this there are talks of ATO detection models.

7. Collections improvement

   Collection directly impacts bottom line. (TPV can be good or bad, blocked users might be true positive or false positive -- collection does not have that dichotomy), so small gains in collections are equivalent to larger gains elsewhere. We continue to improve collection model. Curretly an RL based system is being developed.

8. Support and collections

   We have to explore how we can do better job at support using AI. Once we have made progress at support we will explore investing in AI for collections.

9. Leveraging SMS data (again)

   Due to policy compliance reasons we cannot upload raw SMS to our servers. We need to do the best we can within this constraint to utilise SMS data.

10. Limit update orchestrator

    Many systems are changing user credit limit these days, and they may collide with each other. There needs to be a central limit orchestrator.

11. Continuous improvement of blocking filters

    We need to survey all the filters - and remove or relax the filters which don't help us.

12. Specific work on Billbox delinquency

    We need to pay attention to billbox delinquecy and see if we can reduce it.

13. Continued work on detecting phishing and ATOs

    We need to continously improve our fraud detection capabilities and fast actioning on them.

14. New capability development for fraud prevention

    New product capabilities many times prove effective in curbing fraud. Two examples are alphanumeric OTP and mswipe. We need to develop more capabilities this year - server based linking, auto read otp on linking, and some other capabilities in pipeline.

Improving how we work

1. Rethinking control sets

   Control sets have go unwieldy. We know AFS control sets account for than a proportionate number of blocks - we know the reason and it pollutes our analysis. NU control set users are stuck because of CB. Credit upgrades have a rolling control set. Many experiments have their own control sets. How should we think about control sets?

2. Engineering rigour in DS

   In several parts of DS, engineering rigour is less than needed. Writing clean code, reviews, committing the code, tracking essential metrics - are some areas where we have to double click on.

3. Experimentation framework

   At any given time, many experiments are in progress. Only individual data scientists know how calculate metrics (TPV, delinq, retetion etc) of the experiment users. There has to be a central way to track experiments - a central way to add users to experiments and then tracking metrics of various cohorts. Any metric related to any experiment will then be able to be calculated by anyone who is interested.

4. Iterate faster

   Rate at which we do various experiments should increase. Lot more experiments should be running at any time. Some would succeed and some won't.

5. Models working more in tandem

   Over time models have proliferated. Two models have have differing opinion about the same user and that can lead to suboptimal outcome. Is it that approval model approves but SLAM blocks? If so, we should figure out who to give priority to. Is it true that upgrade model gives upgrade and few days later AFS model blocks? How come there were differing opinion about the user from different models?

6. Rigorous tracking of experiments, models metrics and output metrics

   Three things here. Firstly, we need to plan experiments - pre experiment write up explaining the experiment, and success criterion, and need to track all experiments in central way. Secondly, we need to track model metrics (distribution of feature values), and thirdly track output numbers, like number of blocks, precision of blocks. We have been doing this but I want to be more systematic about this.

7. Global sizing of problem and tracking that

   This is fuzzy in my mind. We need to size up the problem better. How many users we see on daily basis and how many of them have credit lines? How will blocking slightly lesser number of users change MAU? These types of dyanmics are little studied.

Engineering themes

Main projects

1. TS Improvements

   Currently we store all the historical transactions in the table. We need move away from that in following steps
   1. Create billing service - drive it from UEP rather than ledger.
   2. Eliminate other dependencies on historical transactions - updates to UEP itself currently rely on full transaction history.
   3. Archive transactions

2. Rewrite of Pi3 service

   Pi3 is several year old service and has become very complicated.

3. DE - dask and redshift elimination

   We had aimed this for 2024, but priorities changed midway. Can we get this done in 2025?

4. Measuring mobile app stability and app performance

   There are two aspects - stability (number of crashes) and performance (how long it takes for a page to load). We have made progress on the former. In 2025 we need to continue doing former and also do latter.

5. Eliminate appsflyer, clevertap, onesignal

   We pay around 7L per month to appsflyer, around 12L per month to clevertap and 1L per month to onesignal. We need to eliminate these tools from our stack.

6. Better telemetry across organisation

   We started to use new relic, but scaled back due to cost reasons. We will use open telemetry across org to log application performance metrics.

7. AWS cost optimisations

   We work on this wherever we find opportunities. Our current AWS bill is around 1.1 cr a month. We want it to reduce slightly - may be go down to 1 cr by the end of the year. That is a big ask because inevitably new features / components will come up. We need to continuously work on finding opportunities and exploiting them.

8. Organise data so that queries are faster

   This is an idea whose time has probably come. Currently, a lot of computation is repeated across the org. E.g. there is a code (decently high run time) to determine which user is new and which is repeat. It gets run repeatedly perhaps tens of times every day. We have to isolate such cases, and organise data suitable so that queries are faster.

9. DPDP compliance

   DPDP is coming. We will need work to be DPDP compliant.

10. Improve productivity using gen AI tools

    Use LLM based tools to increase our productivity.

11. Better monitoring and reliability of critical services

    Late fee application, bill gen & reminder campaigns, OTP messages

12. DB cleanup

    As on today TS db stands at around 1.7TB. A large chunk of this is transactions and related data, which will get liberated once we start archiving older transactions. But there are some low hanging fruits in there, for example tables like: payment_failures, settlements, one_time_settlements etc, which host truck load of useless data that doesn't need to be there in a critical path db. We need to find which use-cases need historical data in these tables, and host those use-cases, along with this data, separately elsewhere in a cheaper db instance. Shreya's guesstimate is it will still help us liberate around 15-20% of the db size, adding around 6-months of runway without db size upgrades. The challenge is in detailing this analysis and ensuring we don't miss something critical. But cost-benefit advantage is definitely there.

13. Disaster recovery

    If AWS loses our data then we are finished. We need to replicate the most important data to other clouds for disaster recovery.

14. Monitoring of external end points

    Monitoring of externally accessible endpoints to detect any data leaks that may be happening through them.

Other projects

1. Shreyas's wishlist

   • Moving credit limits to another service

     Currently CLs get served from and updated in the TS DB. They don't need to be, since they update once in a day for majority of the users, and don't need a frequent DB hit. Also, even in case of frequent update, say when we have dynamic(ally changing) CLs, serving them from a cached layer (and written/updated through this cache) makes even more sense. So altogether, we should ideally serve and maintain CLs in a dedicated service, deployed on memory-optimised instances, so that they also work as a caching layer themselves, alllowing for efficient and reliable cache invalidations, ensuring no loss of consistency. This is good to have since it's not burning right now.

   • Ledger settlement across products

     This needs solving at the product/business level first, and has been detailed in the tech debt doc here

   • Simpl One Bill

     This also has been pending since a long time. It's not very pressing, but it has always been a good to have since that would give a firmer base to how we reason about billing, cycles and products. It's been detailed here in the same tech-debt doc: tech debt doc

   • DLQ for event reactor

     As of now, due to its primary use-case of dispatching simpl-api-events as triggers for various downstream services as http calls, event-reactor doesn't wait for long in case a downstream service is not available. Post the prescribed number of retries, it moves on. We have been coming across some non-urgent usecases that would do better if ER had a DLQ (dead-letter-queue) that allowed re-processing of the failed triggers at a later time. This is a can we have been kicking down the road for a while, and the usecases are increasing. So the sooner we solve it the better. Challenge is in solving it in a generic manner and without involving bottlenecks like DB based caching that would nullify the benefits of having a high throughput processing channel like kafka.

   • UEP caching

     Since consistency is paramount in Eligibility flows, we always go to the db to fetch the UEP for every eligilbility request. The flows are sleek and postgres' own caching helps, so we don't even notice the number of db hits. But this also will eventually hit some kind of performance bottleneck. We need to progress towards the final destination in the overall plan, that is - serving eligibility from redis. So far we had been blocked by a reliable approach that didn't trade consistency off for doing this. But there is a possible way of doing this now, and we should explore it and take it forward. This is not super urgent, but like all things eligibility, it needs solving when the house is not on fire.

2. Vinayak's wishlist

   • Deprecate Identity Service: Transition to GraphDB as its replacement.
   • GraphDB Alternative: Explore alternative GraphDBs to replace NeptuneDB.
   • CPU Quotas: Implement CPU quotas for ECS services.
   • Domain-Based Kafka Topic Consolidation: Organise Kafka topics by consolidating relevant messages into a single, domain-specific topic. We have a single topic with all the messages now.
   • Common Code Libraries: Develop shared libraries (e.g., DB connectors, Kafka publishers, loggers, error notifiers) to reduce development time, minimise errors, and ensure consistency.
   • Near Real-Time Structured Data: Enable near real-time structured data processing using Databricks. Structured Table DAG in near real-time?

3. Vikas's wishlist

   • Provide experimentation capabilities in sending reminder comms.
   • Create UI for reminder campaigns to configure campaigns/templates from UI.
   • Deprecation of RabbitMQ.
   • Unity Catalog in Databricks.

4. Saravanan's wishlist

   • Achieve a consistent app launch time under 1 second across all supported devices.
   • Maintain 99.9% crash-free sessions through proactive crash resolution and monitoring using tools like Sentry or Firebase Crashlytics.
   • Implement a global performance tracker to measure and optimize app responsiveness and user interactions.
   • Reduce APK/IPA sizes by 30% through optimized assets, lazy loading, and modular libraries.
   • Implement intelligent caching strategies to improve resource usage efficiency by 20%.
   • Regularly test the app with ethical hacking techniques to identify vulnerabilities.
   • Intelligent push notification for iOS to improve the reachability and reduce the sms cost.

5. Manish's wishlist

   • Better tracking of Phishing attacks and metrics in general
   • Enhancing DTM tool
   • Sending filter outcomes and suspected case to monitoring to have constant feedback for filters
   • Improving precision of filters
   • Calling users if there are multiple filter declines
   • Automated calls to suspected fraud cases
   • Bring down delinquency in Billbox
   • Improving product features
   • Explore alternate measures instead of blocking accounts - like Reducing Credit Limit, asking for OTM

6. Farhan's wishlist

   • Bring Total Airbrakes across APPX services under 50 from ~250 today
   • Dynamic / Configurable Home page design from Backend (Ambitious and needs alignment/input from Business and Product)

7. Rahul's wishlist

Finally

Please think of some areas where you want to improve upon. If you are not improving, you are deteriorating. Every year pick some areas where you would want to get stronger.