M-Pesa / Daraja API Credentials

This is one of those articles you write when you have just had it, when like a hundred people keep asking the same question over and over again and can’t seem to understand, or just don’t want to do the work, and want to be spoon fed, or simply just can’t understand what the heck to do about it or go about it. I hope this article clears as much of this as possible.

This is about M-Pesa’s credentials, in the Portal and outside on their Daraja API. And here, the word Credential should be taken to mean any form of code or security that seeks to authenticate the user to Daraja or M-Pesa, no matter the type or format, for the user to be able to access its services. This article is also very specific to Daraja.

M-Pesa basically has 2 sets of credentials:

  • Portal Credentials – these are used by users on the M-Pesa portal, also known as operators.
  • API credentials – these are credentials created for use in API calls to the M-Pesa API via Daraja.

There are other credentials on other older M-Pesa APIs, but they are not our focus here.

Portal Credentials

When you access the M-Pesa portal, you have a user whom you use to log in. That user has a password which you either assigned to them if you created them, or was given by Safaricom. That user is also known as an operator, and more specifically a web operator. That means they are allowed to access M-Pesa via the web portal. Their password represents the first type of security credential for M-Pesa, a simple password backed by a one time verification code for every login. This password is as simple as they come. The only requirement is that it has at least one special character, one capital letter and one digit. Note that M-Pesa does not recognize the ‘@’ character as a special character.

For a web operator, their password has an expiry period, around 3 months (last I checked). That means you gotta keep your password updated regularly, or you will be locked out and the web operator’s status changed to dormant. When that happens, if you are not the Business Administrator (this is a role assigned to the first primary operator of a shortcode), you can have the Business Admin for your shortcode re-enable you, otherwise you gotta get in touch with Safaricom M-Pesa Business team to re-enable you. If you plan to use a web operator for API requests, always remember to renew the password before the expiry, otherwise all requests will fail with the “The Initiator Information Is Invalid” error.

Now, this tutorial assumes you want to integrate into the M-Pesa APIs. And that means you will need an API operator. This is the API equivalent of the web operator, meaning they are only allowed to perform API calls, and can never log into the web portal. Their passwords also never expire (last I checked). They are created by the Business Admin of a shortcode to enable the business owner to perform API transactions. Once created, they are then also known by their API name: Initiators. Thus their assigned usernames become known as Initiator Names and the assigned passwords become Initiator Passwords. Remember you have to assign them the right roles to be able to perform the required API calls.

For the API operators/Initiators, their passwords represent the second level of credentials for M-Pesa. The passwords need to be encrypted using an RSA Public Key certificate provided by Safaricom before being used in an API. The Certificates are available in PEM on the developer portal documentation. Once encrypted, the resulting cipher-text is then what is used as the SecurityCredential or Initiator Encrypted Password in the body of API requests to Daraja and other M-Pesa APIs e.g. in the Daraja API B2C Sample request below, the SecurityCredential parameter contains the cipher-text resulting from the encryption process described mentioned above:

{
    "InitiatorName": "apitest361",
    "SecurityCredential": "iDIurkCG4r9F7lCCNNkctseY9MgiddL6F1VmmJH1DSk5n5U4HPkE3gj1+UFNQ+3MUIzIkR7EgBov9/rqzeXJVb8JeTTD/10XSdTU5eJJeWsqI1VSXpIe/j4xa7HNuezxRJ/KtTWu3bdq9heozyXMn/5ErogjfwIlfQ4Bkhw8+9SEGoNkRA95Idp1PSg/ElPOVnrKuPFTf0HS/iTmuKvVBkY6oAFSObVx7DmqqoL2K4P8ZNh3EDVfQtCQGUnF1OLOab5X46wqVW5p7UJ/5kVFdN8Yuw172VNOjyNILjUXwqqAhS4WKXC28cXd4faYrG5E5AQfGGjNIlYRqWse8w23fw==",
    "CommandID": "[CommandID]",
    .... //other data
}

That is the ONLY place you ever use an encrypted password in API requests to M-Pesa. Nowhere else! Remember that.

Those are the two sets of portal credentials you can expect to work with on the M-Pesa portal side. Moving to the APIs…

API Credentials

When you get to Daraja, you are expected to authenticate your API calls to Daraja (if you don’t know what am talking about please take a detour). Thus authentication on Daraja takes us to the other set of credentials: API credentials.

Note that the API operator/initiator credentials are created on the portal, but used on the APIs only.

When sending any request to Daraja, you need to send a specific header as part of the headers. This is the Authentication header. For almost all API calls to Daraja, you need to first fetch an access token that will be authenticated against the API. To get this token you use the Generate Token API. This API requires you to send the same Authentication header parameter to get the access token but with a different value. That header is an M-Pesa API credential, and is made up of base64 encoded consumer key and consumer secret from an App of your choosing on the developer portal, joined together by a full colon (again, all that is here). That header must always be present in every request, under the Authentication key, and expires after 3600 seconds, or 1 hour. So keep in mind to refresh it regularly. Note that you only encode the consumer credentials, and that you only ever use the tokens in the HTTP header, nowhere else. The screenshot below shows where the header is applied when making an API request from the portal. The <Access-Token> variable is replaced with the result of the base64 encoding described above. Remember to always include the prefix required in the header e.g. the word Bearer in the screenshot, followed by a space, then the token. Otherwise requests will fail with an “Invalid Access Token” error

When used in actual API requests, the header is usually used in the format Authorization: Bearer <Access-Token>. So take note of how to include headers in your selected language / environment.

Last, but definitely not least, we have the Lipa na M-Pesa password. This is a credential that is only used in the body of Lipa na M-Pesa requests. To use or apply this credential, you first need to get a passkey from Safaricom, or if on the sandbox, use the provided passkey. Then create the password by base64-encoding your concatenated shortcode + passkey + timestamp. That gives you the password to use in the Password parameter of LnM requests. Sample usage is shown below:

{
	"BusinessShortCode": "174379",
	"Password": "MTc0Mzc5YmZiMjc5ZjlhYTliZGJjZjE1OGU5N2RkNzFhNDY3Y2QyZTBjODkzMDU5YjEwZjc4ZTZiNzJhZGExZWQyYzkxOTIwMjAwNDMwMTgzOTQ5",
	"Timestamp": "20200430183949",
        .... //other data
}

Note that the Timestamp value in the request must be the same as the timestamp used in the creation of the Password. Also the timestamp must be in the format YYYYMMDDHHmmss. Otherwise you also get the “MerchantValidate – Wrong credentials” error. The format pattern usually differs between languages, thus the base rules are:

  • The year must be 4 digits long
  • The Month should use its numeric value, not name e.g. January is 01, not Jan, January or whatever else you are thinking about.
  • The Day is the day of the month, between 1 and 31.
  • The Month, Day, Hour, Minutes and Seconds must be 2 digits long, prefixed by 0 (zero-padding) where needed.
  • The hour should be 24-hour format.

And that’s the only place you ever use the LnM password.

Thus, in summary, you pretty much always use either one or two credentials when working with M-Pesa APIs. If you ever find yourself using all three, just delete all your code and start over.

If I can summarize the APIs and their usage of the credentials, it would be as below:

APIM-Pesa Initiator PasswordAuthorization Header (Type)LnM Password
Generate Token ✔ (Basic)
Reversal ✔ (Bearer)
Transaction Status ✔ (Bearer)
Account Balance ✔ (Bearer)
B2C ✔ (Bearer)
C2B (Register URL) ✔ (Bearer)
M-Pesa Express (LnM) ✔ (Bearer)

I have tried and created sample codes on how to generate each type of credential for Java, PHP and JS. Unfortunately for JS I haven’t found a way to encrypt the password from the server side, thus the available code only works on Browser side. If you have sample Node code to encrypt the M-Pesa Initiator password, please host on GitHub and share a link, I shall put it here for others to check out:

Again, if none of the above make sense, start here.

Note that the certificate in the repos above is the G2 API certificate, thus will most probably not work with Daraja. Download a fresh copy of the certificates from the Daraja portal to work with on Daraja.

That should give you a good head start in that stuff. Also note the very generous use of the word only for those peeps who use a single credential everywhere they want. Now, I don’t wanna hear anymore credentials questions.

Pay via ProxyAPI on WooCommerce

Hey All! We now have a new Plugin available for WooCommerce Clients and Admins backed by Proxy API!

The new plugin enables one to accept Lipa na M-Pesa payments on their WooCommerce sites powered by Woo Pay via ProxyAPI plugin. The plugin is integrated to use the PVP functionality introduced in the PVP API and this enables all payments made via the plugin to be initiated and completed automatically. That means the plugin also handles callbacks from Proxy API and updates the orders with the current transaction details at the back of it all.

Woo Pay via ProxyAPI (WPVP) Plugin does not keep a separate table for MPesa transactions. It receives and updates transaction results directly on the Order, allowing for a more convenient way to keep track of orders. All MPesa transactions are directly tied to their orders via the Transaction ID, which allows an Admin to easily see as soon as they open the order on their administration portals. A sample output is shown below:

Note the M-Pesa Transaction ID directly tied to the order on completion after the Payment Type description

WPVP also enables the transaction to be directly tied to its CheckoutRequest ID as received from LnM API, and one can use this to query the status transaction later if the transaction is not automatically updated within a few moments after a user responds to the Safaricom payment prompt. The callback also attaches any extra data received from Safaricom to the order. All this data is easily viewable from the order metadata as shown below:

Metadata of the transaction received from MPesa.

WPVP also has custom actions which a developer can take advantage of to initiate requests after either of three events have occurred:

  • When a payment request has been sent successfully to LnM API. This event is named ‘proxyapi_pvp_payment_pending‘ and has four parameters:
    • The orderId for the current order
    • A unique RequestID created by the Plugin for the current request
    • The MerchantRequestID and…
    • The CheckoutRequestID, both received from LnM API after sending the request
  • When a payment callback has been received, and the callback indicates a failure. The event is named ‘proxyapi_pvp_payment_failed‘ and has three parameters.
    • The orderId
    • The ResultCode and…
    • The ResultDesc, both received from LnM API when a transaction has failed.
  • When a payment callback has been received, and the callback indicates a success. The event is named ‘pvp_on_payment_complete‘ has only one parameter, the orderId of the successful request.

This events will allow you to hook into the payment process and perform any other events after the order has been updated accordingly. Some sample code on how to use the above hooks is available in the Plugin Repo on Github.

FYI, an order is automatically set to failed when an error code has been received from Daraja, and automatically set to completed when a transaction ID and confirmation callbacks have been received from either Daraja or Proxy API. There is no manual action needed for either of the two and no manual checks of transactions required either unless the transaction has not been updated for more than 5 minutes, indicating LnM API was unable to send an error callback back to the plugin.

Note that the Order Number is also used as the Account Reference when the LnM request is sent to M-Pesa. The user’s phone number is also mandatory, thus beware not to remove or skip the phone number field during checkout.

On top of it all, you also get the benefit of using Proxy API portal to confirm a transaction was sent and the callback also received if in doubt.

And v2.0.0 now has a new custom M-Pesa Report tab under reports to show you the latest received transactions from ProxyAPI.

A sample WooCommerce application showcase can be found here

For details how to on-board to Proxy API, check out the documentation

Get it on WordPress!

Pay Via Proxy API

Do you have an online business, store or website?

Do you have a Safaricom Till Number or Paybill Number?

Do you wish to integrate into M-Pesa or Lipa na M-Pesa and receive payment notifications automatically from your site?

Well, Proxy API has the right service for you!

We have developed a new integration named Pay via Proxy API that enables merchants and developers with online stores to easily integrate into Safaricom’s MPesa. The setup involves simply integrating a single button on your site that enables the client to initiate a Lipa na M-Pesa request from your site. By clicking on the button, a new window is opened, the client enters their phone number and an instant payment request is sent to their phone to complete the transaction. Simple, right?

Sample PVP LnM Request

This integration also combines Lipa na M-Pesa API and C2B API whereby the 3rd party/merchant or developer receives two callbacks: one from from Daraja and the other from Proxy API; or at least one callback from either, on successful completion of the request. This enables one to have more stability on their system and not have missing callbacks sometimes seen when using the Lipa na M-Pesa (LnM) API alone. This also means, for any unconfirmed callback, the 3rd party can check for the missing transaction on the Proxy API portal, similar to Query Checkout API on Daraja. This is a feature that will be integrated soon.

After initiating the request, the 3rd party can expect either one or both callbacks from Daraja or Proxy API. The callback from Proxy API comes from the C2B integration that will be automatically handled by Proxy API. This enables one to crosscheck both the LnM callback and Proxy API callback for the details of the transaction, whichever comes first.

Check out more info on the documentation available here.

Then begin onboarding here pap! 🙂

Safaricom Tills API Integrations

To all merchants and developers out there who own or have Safaricom Tills and who wish to integrate into Safaricom APIs, this is for you.

If you wish to integrate into MPesa API, you cannot use a Till number alone.

Tills are like small kids or even babies around money. Ideally, people can give babies and small kids any amount of cash they wish, but they do not have the facility to store it themselves. That is done by the parent of the child who has access to an account. For Safaricom Tills, that parent is a shortcode known as the Store Number. It holds the accounts where all the cash paid to a Till is deposited into. It is the shortcode which can access or be accessed by APIs. Tills have no accounts of their own, and don’t have access to any portals, thus you can’t log into the MPesa Org. Portal using Tills. Only the parent Store Number has those. Tills only act as quick facilitators of cash collection for the Store Number and also for organizational purposes.

Note also, there is a 1-to-1 mapping between the Till and the Store Number. One Till can only have one parent Store Number, and one Store Number can only be mapped to 1 Till

So make it a point, when applying for a Till Number, to also request for the Store Number and apply for an Administrator for that Store number, if you ever plan to integrate into APIs in future. That Administrator will be used in the Go Live process on the APIs and also to access the MPesa Org. Portal.

Note, there is also another shortcode above the Store Number known as the Head Office number, or H.O. for short. This is at a “grand-parent” level to the till, and can never be used for APIs, as it is a grand-collection account for the overall parent organization. Money collected by the Store Numbers via their Tills is propagated upwards during settlement processes e.t.c. One H.O. can have multiple Store Numbers under it e.g. those large supermarkets use that hierarchy to map out tills to a parent/main headquarters somewhere in Town, and whatever those guys earn, can be propagated upwards to the headquarters’ H.O. This is most probably owned by those guys who give you the Till numbers, so don’t even bother checking it out.

So, when you are using the MPesa API, whether Proxy API or Daraja, during Go Live, you should have access to and utilize the elements of the Store Number, not the Till.

Security on M-Pesa and Proxy API

Here is a topic which, I think, everyone wants or needs to hear/read about, but few, if any, have touched on….

Security on M-Pesa and its APIs.

Am pretty sure most owners of Safaricom Shortcodes want to know how to secure their M-Pesa accounts and Internal Systems against malicious access or API calls by either insiders or external forces. And am pretty sure there are cases of organisations losing money to some scrupulous insiders, developers or tech-savvy people via API access e.g. peeps paying themselves or transferring cash to weird numbers using illegal B2C API calls, people transferring cash via B2B to their bank accounts, or people listening on C2B callbacks for shortcodes they do not own or even sending their own C2B validation/confirmation requests to a 3rd party system for malicious purposes. It is actually very easy to lose millions in a single moment if there is no thought paid to security on your systems.

This document will be a living document, changing to keep up with new ideas and feedback from fellow developers, shortcode owners and other parties. It shall seek to provide guidance as to the ways in which one can protect their systems, APIs and M-Pesa accounts from fraudsters and other possible malicious parties using existing API capabilities.

This article is not an expert’s opinion on how to achieve security on your system, for that, consult a Security Expert. And neither is it the one-stop-shop for all security issues one might face on their systems. The article will not go over common sense issues such as using SSL on your system. FYI if your system is not using SSL for all API requests, I don’t know what the heck you could be looking for here, you should be on Stack Overflow, somewhere else or even Google

This is simply a guide specific to M-Pesa showing the techniques one can use to secure their systems against those who would want to use M-Pesa to gain an unfair “monetary” advantage for themselves.

The document will be split into 3+ sections, covering security from the M-Pesa portal itself, to the developer side from the 3rd party’s perspective, down to the APIs themselves. Each section will have points showing some practises you, as the developer or the shortcode owner, can adopt on your end, whether you are tech-savvy or not, that can assist in keeping your virtual cash safe. The sections will probably be added to as more info comes in

Lastly, the article will probably be veeery long… and will continue growing 🙂

Beware, they are also reading this, and will always look for ways around your system
Credits: Clipart Lib

M-Pesa Portal

Charity begins at home… or so they say. So same applies to Security. It all starts with the Baba Yao (parent) of all those systems and how you work with it: the M-Pesa Portal

  • For the Business Administrator, know the power you wield over every operator on your system. You can add and disable operators at will, reset their passwords, change or remove their roles, and generally have SU-like powers over them. You are the first gateway into the security realm on M-Pesa for your organisation.
  • Before login into the M-Pesa portal (for all web operator roles), you have to apply for and receive a certificate to log into the site. That certificate is quite troublesome to move to another machine, if at all possible. Use that as a first line of defence to limit access to your Portal. Only apply for and install the certificate on a specific machine which you have full control over. For Business Administrators, only ONE person should ever have access to their credentials and machine. If anyone else touches the Business Admin’s machine, then there’s bound to be issues, which grow with the size of the organisation.
  • For every user the Business Admin creates, they should aim to only have a single human being per Operator and if possible, to track the machine on which their respective certificates are installed, and keep a close eye on each. M-Pesa has great audit logs, which, combined with access control to the machines, can assist to track exactly who did what on the portal and where they were.
  • To learn how to create Operators, use this document.
  • Know the difference between a web operator and an API operator. a web operator is created to only access M-Pesa via the portal interface. An API operator is designed to only access M-Pesa via API calls. There is a reason for such a design.
  • You should NOT assign permissions designed for an API operator to a WEB operator. This opens up the system to quite a sizeable can of worms. A single operator type will hardly perform any risky stuff on the API, but combined, they can do wonders (believe me). An example: imagine a single Business Manager (web operator) with API permissions overseeing an organisation’s finances. Lets first assume he’s on the good guys side. He can view all transactions, check all reversals initiated, and even see all outgoing B2C payments (for B2C accounts) or incoming C2B payments. Now let’s add some well known human greed, and let the shortcode be a B2C one. The guy’s operator also has API capabilities, thus employs a developer to make a secret separate system that can make API requests, using his credentials. Now, the guy begins sending separate 1 shilling requests for every transaction that goes out via B2C (stupid idea, but still shows my point). Assume the company has like 1,000 transactions per day. If that guy has the only visibility of the M-Pesa account in the organisation, he just earned himself a cool 1K just by sitting on his desk. If he’s even more daring, he could simply join up with random guys on the outside, add them to his “payroll”, and use the same technique, though now using bigger amounts and describing them to be the same as the legal payouts. Outside, the other guys simply give him his cut. Using the same 1000 transactions, each being given out at least 10 bob, and using like 20 separate guys, each with at least one active M-Pesa SIM card (probably opened using fake ID), that translates to around 1000*10*20 = KES. 200K. That means each guy gets KES. 10K, each within the M-Pesa limits, and depends on how good the guy’s network is. If the Business Manager gets 40% of each, he takes home 4K multiplied by the number of “employees” he recruited. He doesn’t even need to be in the company to do such. He can simply set up such a system, find an ideal time to leave the company, ensuring the credentials remain the same, and initiate the same plan when outside, leaving the blame to whoever will be running the show on the inside. And when outside, he can actually increase the per-person share to KES. 30… 50… 100… you get the point. Now imagine some of that innovativeness we have been seeing around (e.g. people digging underground tunnels to banks) applied to such a situation and tell me what that person cannot do. Basically, let API operators remain API operators, and the same to apply to Web Operators.
  • If you can, employ the maker-checker capability. This is an M-Pesa capability that allows one web operator to only initiate a transaction, and another separate operator on the same or higher level to complete the transaction. This is used for mainly reversal operations, and ensures no single person has power over the whole transaction. All parties need to collude to actually pull anything off. But if you have API permissions, that bet is off the table.
  • Business Admins, always check on the web operators regularly, and disable operators whose users are not around or are on leave. Someone out of office should not be allowed to still access the system yet they are supposed to be enjoying themselves elsewhere. Let them be enabled when they come back.
  • For integrations, I would suggest creating specific operators for that, and another set of operators for the live system. This is going to be explained further in the next section
Sound familiar??

Developers

So, you got a business, and want people to pay you. Cash is one way, the other great option is M-Pesa, since almost everyone is on it. Then M-Pesa guys just give you a SIM card and tell you that you shall receive notifications via SMS. But your business is an online business.

They then inform you they got an API for automated notifications. Great, you can now automate your payments! But wait, you are not a software developer :/. So now you need to get a developer to work on your system.

Now the developer needs access to M-Pesa to integrate into your systems. Safaricom has provided a Sandbox to test with, that’s great! But sooner or later, the developer will need to integrate the live API into your system too. And that means somehow, they shall need live credentials. And that get’s your mental gears rolling… If they get access to your system, will they use it to pay themselves if I don’t pay them on time? Will they use the access to make their own little vampire system to suck the money out as it comes in (especially those who use a C2B/B2C combined shortcode)? Will a third party API provider like Proxy API use those credentials to pay themselves on the side? How will I know when something weird is happening on the system? Can I stop them if so?

Well, hopefully, following as many of the below tips as you can, you could be well into making your system’s M-Pesa security manageable, if not top notch.

  • First, read the previous section if you haven’t, then come back here
  • Next, for business owners and the non-tech-savvy team, in software engineering, we have a concept called Version Control, implemented by a Version Control System, VCS in short. It is basically a system that manages changes to pretty much any collection of information, but we shall focus on Software collections for this tutorial. That means you can use the system to manage multiple versions/copies of the software, its documentation and configuration. This, I believe, should be your first step: integrating VCS capability into your system during development. It should be one of the foremost goals, extending past security and going into the realm of disaster recovery and multi-version software management. From a security perspective, the single most important use for VCS is separation of configurations from actual source code. That means you can create a system which picks its secrets and credentials to use on a specific system from a separate file or location within the same server/node, or even a geographically separate server e.g. a developer can create code for you from Nairobi, test using their local machine’s Database, simply host or “push” the code on a shared server, ping you in Mandera, you fetch or “pull” the version that is selected to run, set the location in the config file to Mandera’s live DB (even better, no need to change it if you already had the service up and running), and run immediately, with no fuss. This centralization, combined with package management systems, minimizes and even in some cases eliminates those “it was working on my machine!” scenarios. Examples of popular VCS systems you can check on include Git, Mercurial and Subversion. From the above, first suggestion would be: design your system to have its configurations separate from its source code. The configurations will include anything from database URLs, usernames and passwords, API endpoints (because these can change e.g. moving from sandbox to staging then to prod), and API credentials e.g. consumer keys and secrets. Let there be a separate file on the system or separate system with an extremely low latency, which shall hold all those credentials. Then ensure only the minimum access to those resources e.g. only apply the 0600 permissions octal for all configuration files, and then set them outside the publicly available base folder. This will ensure that only the user with whose role the system is accessed can read the file, and no one else, except the SU. If on a separate system, lock down on network access to that machine holding credentials (more on that later). And that assumes the root user for Linux/Mac, or Administrator for Windows, is actually well protected too. Otherwise all the above is for naught. This will ensure that when code is being migrated between environments, there are no secrets hard-coded into the system, which can be used as weak points to bypass authentication or authorization, or be used to launch unknown transactions without the owner’s knowledge. It also ensures that any code being moved only carries its generic/universal logic to perform whatever it needs to do, and only fetches environment-specific variables from the external resource. Here’s an example of how to set up such on Linux. Also, if you can afford it, consider using Docker for your services, it comes with an inbuilt credentials management system that can be incredibly useful for such situations.
  • So, your developer has managed to separate code from config, then what next? When integrating into M-Pesa APIs, a developer is first assigned test credentials to test their system with. This enables then to test out their setup and credential encryption/encoding/hashing schemes. Based on that, a developer should not be allowed to use static credentials on the system. The developer should learn to apply whatever encryptions/encoding/hashing/other actions necessary on the credentials and set up the system to always apply them dynamically during runtime. For. Every. Request! This enables the Administrator to change their system or operator credentials anytime without worrying about system breakdowns when old static credentials are still being used in the code, necessitating another “deployment”. This also means the codebase can be changed anytime without worrying about credentials to external systems. Static credentials also means the developer will most probably leave having set up the live credentials, and leave the project or company with that knowledge in mind. That also means there is a high risk the same developer can use those same credentials to make API requests on their own via another system, especially on e.g. Daraja, where any IP address is technically allowed to make API calls as long as you got the right Consumer Secrets and Credentials, and M-Pesa Operator credentials.
  • Then since the developer has access to test credentials, the developer should not be allowed to use live credentials to test/integrate the system. A good developer should understand how the credentials are being applied, and develop the system to allow a “plug-in” kind of scenario for environment variables and secrets, meaning if the Admin decides to test on sandbox, the config location is simply modified to point to sandbox credentials, and when done, a quick pointer back to production credentials will be done, and the system should continue purring along, being none the wiser as to what just happened.

Here’s an example I use on Proxy API. I have 3 environments: “local” for a developer’s local machine, mainly for innovating, inventing, adding and testing new stuff, “staging” for an environment pointing to production, but only accessible by a limited set of people to test new changes on production; and the “production” environment itself. So, for each environment, I have an .env file, which carries all possible variables that change in between environments. These include API and Application URLs (including callback URLs), DB usernames, keyfile locations and passwords, and API credentials. A sample .env config file section is as shown below for PHP peeps on Laravel or SlimPHP (of course the credentials are fake :P):

... //other configs

//example on how one can set the base URL for staging and local dev.
//Uncomment only one
BASE_URL="https://staging.example.co.ke"
//BASE_URL="http://localhost"

API_VIEWS_PATH="/path/to/views/for/current/environment"
API_LOGS_PATH="/path/to/logs/"
API_OPENSSL_ID_KEY="opensslkey"
API_OPENSSL_KEY_FILE="/path/to/keys.csv"
API_OPENSSL_CIPHER="aes-256-gcm"

HTTP_VERSION="1.1"
RESPONSE_SIZE=4096
OUTPUT_BUFFERING="append"
DETERMINE_ROUTE_BEFORE_MIDDLEWARE=1
DISPLAY_ERROR_DETAILS=1
CONTENT_LENGTH_HEADER=0
ROUTER_CACHE_FILE=0

// DB for transactions and transaction processing
DB_HOST="example.co.ke"
DB_PORT=5456
DB_USERNAME="username"
DB_PASSWORD="password"
DB_NAME="staging"

// DB for sessions and logs
METADATA_DB_HOST="staging.example.co.ke"
METADATA_DB_PORT=5455
METADATA_DB_USERNAME="meta_username"
METADATA_DB_PASSWORD="meta_password"
METADATA_DB_NAME="metadata"

GOOGLE_CLIENT_ID="clientid.apps.googleusercontent.com"
GOOGLE_PROJECT_ID="google-project-id" GOOGLE_PUBSUB_LOGGER_TOPIC="my/topic"

With such a config above, I next use a framework that allows one to fetch external configs, in my case SlimPHP (also applicable to application.yml files + Environment variables for Spring Boot, dockerfiles + docker secrets for Docker e.t.c.), and load that config file into memory. Note: since almost all web servers or application servers have a base folder from which they serve a site’s or apps files e.g. public_html or www/html for well known PHP sites, the above file needs to be kept in a location outside that base folder and assigned permissions to only allow the user whose profile or role is used to access the site, to read the file (remember the 0600 octal?). That means, in case of a site breach not involving the SU user, only the files under the base folder will be affected, and any file outside the base folder will not be touched. That assumes file permissions are also well set in that host, a topic outside the scope of this article. With such a setup, a developer can create copies of their own local version of the above config file, have them given to the respective system admins of each environment, and let the Admins configure them appropriately according to the environment. Then you can simply have your code being pushed into a VCS repo, pulled into the respective environment, and simply continue running if they allow, or restart the service if configs are loaded into memory. That also means, the Admin can set their own credentials without the external developer knowing about them, and can change them anytime they wish.

Back to the points:

  • If you really need to give the developer access to live credentials, I would suggest the following: since the aim of integrations is to test the credentials work as expected, and that the operator can actually make API requests, create a separate API operator specifically for integrations, and assign them non-transactional permissions e.g. let the operator have transaction query or account balance permissions only. With that, you as the owner/Admin can learn how the credentials are created, how they fit in with the developer Application being created, and confirm the operator credentials are being encrypted/encoded properly by actual API requests to prod. And when the need arises to use transactional API requests, you can have the developer use the above external config example to create a separate variables file with the necessary configs, show you or an Admin how to use them, and then set them personally after creating the operator specifically to perform transactional API requests. That leaves only operators who cannot perform any transactions on the outside, and whom can be disabled anytime, especially on Going Live, since even data leakage from one’s systems is not something anyone wants.
  • After a system or application has been created, check the logging framework. Ensure no logs expose any secrets or sensitive information. A developer could have placed a simple “echo” to file to confirm what password is actually being passed into the system, and forget to remove it (either on purpose or just by mistake). With those logs visible to everyone with access to log processing systems, it will be quite easy to get an API operator’s credentials and use them as one wishes.
  • Then, once a system is going or has gone live, and you have ensured everything works, prepare to fence the system against all external network attacks. A malicious developer could leave a service listening on a specific port in your Application, and if that App is not firewalled off, the developer could trigger API requests from remote locations to your app, and cause your app to fire off its own API requests to MPesa or other systems. This also means there is a probability a developer could have left a backdoor into your system to get in any time they wish. Once Go Live process is initiated, ensure that all network requests to the Application server to initiate API requests only come from machines within your intranet, or only from specific machines within the intranet. Also, for callbacks, ensure only a subset of machines are allowed to receive the callbacks via network segmentation and Firewall rules. This ensures no callbacks leave the Intranet in case the App is sent to proxy requests or leak some information to an offsite location, and only specific, well known and planned for nodes/servers receive the callbacks.
  • After Go Live, ensure that all API operators used to integrate the system and Go Live are disabled, and apply the same for web operators if you can, since one can easily create new operators for the same. Then, assuming you used the above techniques, create a new set of credentials to use for production, apply it to your prod environment, firewall off unwanted access, and you are good to go.

Hopefully, using the above in combination with what your security experts suggest will provide a good first defence against unwanted access to the API.

API Requests

“How to secure API calls”. Quite a number of questions arise from that single phrase. How do I know it is Safaricom calling my endpoint…? How do I confirm that the callback’s data has not been tampered with…? How do I detect that API calls leaving my App or System are legit…? How do I know that the callbacks am getting are actually a result of my system sending valid requests…? and so on and so forth

Well, as of now, the current version of Daraja has limitations on how one can protect themselves from such. I shall put here suggestions which shall use the current iteration of Daraja and Proxy API to try and protect oneself from malicious external API tampering.First, from within your own system:

First, from within your own system:

  • This is an idea supported by developers also working on securing their own systems from external attacks: for all 3rd party initiated requests, all requests should have a custom parameter appended to the request, whether in the body or in the callback URL, and, if possible, coming from a separate system other than the one actually making the API call to MPesa. E.g. in a financial system, the API developed to make API calls to MPesa should not be the same generating that custom parameter, it should be the financial system passing in the parameter to the API calling system. This will enable the following:
    • Tracking which requests are actually valid, such that one can check the requests received by M-Pesa against the requests made by the API. This can be done using the Remarks parameter or using a query parameter in the callback URL e.g. https://example.com?uniqueid=fwkjefwizxkn. This enables all callbacks to have a reference to the initial request which can be tracked automatically on the callbacks
    • Confirmation that the callback actually came from Safaricom by checking the request parameter or query parameter against valid parameters recorded before the request was made. Unless you are sending plain HTTP requests, or have systems in between that compromise the integrity of your requests (e.g. MITM scenarios) , you are sure the only two parties with access to that parameter are your system and M-Pesa.
  • Proxy API allows the third party to actually have reference data as part of their requests, which are appended to the MPesa request as ReferenceData parameters in the final request to M-Pesa (well familiar to those using XML on Broker). These are also passed back to the user on the callback, allowing them to actually trace the origin or the characteristics of the initial request.
  • Proxy API also allows the 3rd party to initiate requests using their own request IDs, thus, using the idea given above, one can be sure there shall be no replay of API requests to MPesa, as the same request ID from the third party will NOT be accepted by MPesa again. This should be applied with the suggestion to have a separate system creating the request IDs to enable end-to-end tracking of API requests for the organization.
  • For C2B transactions, the main way available to confirm that the requests actually came from MPesa are limiting the callback URLs to only be accessible from Safaricom IP addresses. One can try to request them from the API support team, and place rules in their firewalls or web server configs to only allow callbacks from those specific IP addresses. But of course it does not protect against IP address spoofing, so put extra security measures to detect/prevent such.

A simple example on how to limit a URL to specific IP addresses using .htaccess is given below:

RewriteEngine on
RewriteCond %{REMOTE_ADDR} !123\.45\.67\.[8-9]
RewriteRule ^/transactions/c2b/callback - [F]

The second line is a regex that will match any incoming request from a host with an IP address outside the IP address range 123.46.67.8 – 123.46.67.9, and the third line is a command stating that any IP address matching the previous line trying to access the resource(s) at /transactions/c2b/callback should have their request denied. That can be used to limit the IP addresses accessing your callback URLs to a specific range. An even simpler config to achieve the same as above for latest Apache servers (>= v2.4) would be:

<RequireAll>
  Require ip 123.45.67.8
  Require ip 123.45.67.8
  Require all denied
</RequireAll>

That can be placed in the specific path or folder /transactions/c2b/callback. I believe it’s self explanatory, but is much wider and needs some testing of course. You can Google how to add path specifiers to it.

  • Once that is done, the Admin is advised put up an audit system that checks regularly the number of transactional API requests sent on their system vs the actual number and nature of transactions present on M-Pesa. This can be done by integrating into the SFTP dumps program which receives a file with a collection of all transactions made to or from the shortcode, or using the dump to CSV capability already available on the M-Pesa portal, and using those dumps to cross check against all API requests sent by the API application. This can actually be automated if the system is well designed. This will give them the capability to audit their system anytime and confirm no malicious requests were made nor invalid API calls received by the callbacks.
  • Proxy API will soon implement signed C2B callbacks, where each request sent by the Proxy can be verified against a signature prepared by pre-shared credentials. Due to the high CPU usage nature of cryptographic operations, we are looking for the best way to enable fast verification of callbacks without wastage of node/server resources.

Using most, if not all, of the above tips, we hope you have all the information you might need to create a highly secure MPesa payments system. We also invite you to check out Proxy API and view the array of services available on it that make your work easier. The above document will change frequently depending on reader and developer feedback, so make sure to check in once in a while.

M-Pesa B2B API

To all developers and shortcode owners…

M-Pesa B2B API is now available on Proxy API!

It currently supports Business Paybill, Business Buy Goods, and Business to Business Transfer command IDs.

It will be charged starting at 0.1% of the transaction value, the lowest you can find anywhere, and decreases as the value increases when specific criteria are fulfilled. Imagine that!

Find out more in the Docs.

Proxy API

So, we have been working on a neat piece of art, an API for M-Pesa, named Proxy API.

And we are proud to announce it is ready!

For quite some time, Developers have been trying (and mostly succeeding) in integrating with the First and Second generation M-Pesa APIs (G1 and G2 respectively). It was quite the effort, requiring VPNs and knowledge of SOAP/XML for you to work with the APIs. Well, they are actually great APIs, the most stable,and the most powerful. as long as one was able use them.

Next came Daraja, their new REST/JSON API. Developers were happy that they finally had an easier option to work with, and most decided to try out the new, better option. It was great at first…

Then the issues began.

Most of us have seen complaints of missing callbacks, long Go Live processes, Forms to fill in and upload (in this time and age, imagine that), weird terminology used on the Documentation and API calls, no way to confirm what URLs are registered on their end e.t.c. That list can go on and on. So we decided to do something about it. We decided to create an M-Pesa API that could pretty much fix most of all issues. And that’s where the idea of Proxy API came from.

Proxy API was so named because it was designed to be literally a proxy: simply receiving requests, reformatting into the required XML format for M-Pesa, and forward the request to M-Pesa. But with that option came the capacity to do a lot more.

First, the API was designed to communicate directly with the M-Pesa Broker API, the most stable API and direct entry point into M-Pesa. Secondly, we decided to use REST/JSON to implement the Proxy API interface. This would make it easier to integrate into the API. We also decided, using the knowledge gained from working with Broker, to simplify the API interface facing the developers. Another feature we added was tracing; we could trace requests from developer to M-Pesa and back, then a developer and even business could see what happened when they made a request. Developers could even see the status returned by their services once a callback has been returned or their endpoints called by the API. This would give the developer more control on their integration process and increase the speed with which they integrated into the system.

Then, with tracing requests, came the capability to keep records temporarily for clients to confirm which records or callbacks reached them or not. Clients could place requests to have undelivered callbacks saved and fetched later on demand. Well, this was as long as Broker actually called us too :-).

Thus, the new Proxy API provides the following features:

  • Real time logs and callbacks view
  • Instant C2B callback URLs (Validation and Confirmation URLs) modification
  • Intuitive REST/JSON API interface
  • Live support on Telegram, no emails if you want quick answers
  • No Go Live forms to fill or process, simply confirm ownership of your shortcode via an API call
  • Callback URL testing direct from the Portal

We also have more features in the pipeline for all clients, all building on the simple capability of proxying requests. Please check out the new API at
https://api.proxyapi.co.ke. The documentation for the API is at
https://docs.proxyapi.co.ke/v1/ and we even have an openapi/swagger specification at
https://docs.proxyapi.co.ke/v1/swagger/ .

Hope you enjoy it!

Co-op Bank Kenya API

So The Co-operative Bank of Kenya Ltd. seems to have unveiled a developer portal and exposed quite a number of APIs to work with their back-ends. These include:

  • Account Transactions
  • Exchange Rate
  • Internal Funds Transfer
  • Pesalink
  • Transaction Status Query
  • Account Balance
  • Account Transactions

Check it out at https://developer.co-opbank.co.ke:9443/store/

Interesting times coming up… 🙂