PyInstaller Extractor updated to v2.0

PyInstaller Extractor has been updated to version 2.0. This has been long overdue. The project is now migrated to GitHub and all further development will take place over there.

https://github.com/extremecoders-re/pyinstxtractor

Version 2.0 includes support for Python 3.7 and above. Earlier, the script used to generate invalid pyc files when extracting exe's generated from Python 3.7. This was because from Python 3.7 onward, the pyc header format slightly changed.

Also starting from this version, I am moving away from traditional version numbering. Instead you can always find the latest version on GitHub.

(Finally) Solving the Weasel keygenme

Back in 2016, kao posted the weasel keygenme on Tuts4you. It consisted of two parts - a custom VM in C# and some crypto. The VM implemented the main logic of the keygenme. In my previous writeup about the challenge, I succeeded in partially solving the challenge by devirtualizing the VM. The crypto part was left unsolved. This was mainly because the crypto logic was way too complex and without special algorithms there was no way to have a go at it. As described in my earlier post, even SAT solvers have no luck in cracking the crypto.

However this time after nearly 4 years I am happy to say that I did manage to break the crypto. In this blog post I will describe the process to solve this keygenme along with all the failed attempts.

Initially I planned to post the write-up on this blog but the blogger platform is not too good for handling equations and mathematical terms. Hence I have left the write-up as an IPython notebook on Google Colab which has been embedded as a GitHub Gist below. In case the preview below doesn't load properly on your browser you can always find the original notebook on Colab.

https://colab.research.google.com/gist/extremecoders-re/cc268753b6d25017fe0cff7299ad9be3/-finally-solving-the-weasel-keygenme.ipynb

The keygen can be found on GitHub

Go-dispatch-proxy: A load balancing proxy written in Go

This will be my first post of 2018. I've been very busy these days and didn't have the time to devote to this blog, that was the reason for the prolonged absence of activity. Anyway, in this post, I want to present a tool which I've been working on for some days. It's not related to RE or something fancy. It's just a tool to load balance network connections across multiple interfaces.

To cut a long story short, I have multiple internet connections on my Windows desktop. I needed a tool which would help to utilise all those connections effectively to give an improved speed. In case if multiple connections (i.e. routes) are available, the default behaviour of Windows is to select the one which has a lower metric. If all routes have the same metric, Windows will select one and ignore the rest. [To change the interface metric manually have a look here]

To workaround this problem we can use a load balancer proxy. The purpose of the proxy would be to distribute our connections across the available interfaces leading to increased throughput. There is a project on GitHub - dispatch-proxy which is based on the same idea. It works but I dislike the fact that its written in Node.js. A seemingly innocuous command npm install can wreak havoc on the hard disk by creating tons of tiny files and fragmenting the file system in the process. Naturally, I have an aversion towards desktop apps which are written using node and friends.

Based on the idea I've written SOCKS5 load balancing proxy in pure Go with no additional dependencies.

https://github.com/extremecoders-re/go-dispatch-proxy

The binary consists of just a single file which you can get from the appveyor CI server. It works on Windows and fulfils my purpose. It has not been tested on macOS but will likely work. However, it won't work on Linux. This is because the tool works by changing the source IP address for the outgoing packets. On Windows changing the source IP makes the outgoing packet to use the corresponding interface. On Linux, the packet is transmitted according to the kernels routing table and specifying the source IP has no effect.

To conclude this short post, here are a couple of screenshots showing the tool in action.

Listing the available interfaces

SOCKS proxy at work

Reversing a PyInstaller based ransomware

Occasionally, I get questions about how to unpack PyInstaller executables using pyinstxtractor, how to identify the script of interest among the bunch of extracted files etc. In this post, I intend to cover all of these. Let's get started.

The file for our purpose is a recently identified ransomware having the following SHA256 hash.

Sample Hash: 53854221c6c1fa513d6ecf83385518dbd8b0afefd9661f6ad831a5acf33c0f8e
Download from Mega (Password: infected)

Preliminary Analysis

The executable "hc6.exe" has the following icon. The icon itself is a tell-tale sign that it's a PyInstaller executable.

Figure 1: Icon

Another way we can identify such files is by dropping it in a hex editor. A PyInstaller generated executable has many strings referencing python, towards the end of the binary.

Figure 2: Strings

A PyInstaller executable consists of two parts - a bootloader and a zlib archive appended to it as an overlay. The purpose of the loader is to set up the Python environment for running the application. This includes loading the Python DLL from the filesystem or from memory when the DLL is bundled within the executable. After going through a series of operations, finally it executes the main script and the control is transferred to user code. The loader also sets up hooks for resolving imports which are embedded within, the details of which are beyond the scope of this post. You can refer to the source for more information.

Extracting

Knowing that the sample is a PyInstaller generated executable we can proceed to extract its contents using pyinstxtractor as shown in Figure 3.

Figure 3: Running pyinstxtractor

The latest version (1.9) of pyinstxtractor shows which scripts are the possible entry points to the application. These are the python scripts which are run when the application is launched. Naturally, we want to begin our analysis from here. In this sample, it has identified  pyiboot01_bootstrap and hc6 as the entry points. Among the two, the former is PyInstaller specific and not of interest. The other one named hc6 does sound interesting. Let's have a look at the contents of the extracted directory before analyzing the file hc6.

Figure 4: The extracted contents

Within the extracted directory we can see a bunch of stuff - DLL files, Python C Extensions (PYD) and also a sub-directory out00-PYZ.pyz_extracted. This nested sub-directory just contains compiled python files (PYC) as shown in Figure 5. The pyc files are from the standard python library or from a 3rd party library such as PyCrypto. Hence, in this sample, we can exclude these files from analysis.

Figure 5: pyc files inside the pyz

Decompiling the main script

The main script or the entry script is named hc6, let's have a look in a hex editor.

Figure 6: hc6 in a hex editor

This does not look like python code, does it? However, this was not the case in earlier versions of PyInstaller, where the main script was left as-is, in plain text. Recent versions, compile the py source to bytecode before packaging it in the executable.

We now want to decompile this bytecode file back to python source, however, in its present form a decompiler wouldn't recognize this as a valid pyc file. The reason for this is that the magic value (i.e. the signature) is missing from this file header. A Python 2.7 pyc file begins with the bytes 03 F3 0D 0A followed by a four-byte timestamp indicating when this file was compiled. We can add these 8 bytes as shown in Figure 7.

Figure 7: Adding the missing header

With the above changes, we can now feed this file to a decompiler such as pycdc. In case you do not want to compile yourself, I have provided precompiled binaries at AppVeyor. Decompiling we get back the source.

Analyzing the ransomware

Finally, we can have a look at the ransomware in all its glory. It encrypts files from the following list of extensions.

.txt, .exe,  .php, .pl, .7z, .rar, .m4a, .wma, .avi, .wmv, .csv, .d3dbsp, .sc2save, .sie, .sum, .ibank, .t13, .t12, .qdf, .gdb, .tax, .pkpass, .bc6, .bc7, .bkp, .qic, .bkf, .sidn, .sidd, .mddata, .itl, .itdb, .icxs, .hvpl, .hplg, .hkdb, .mdbackup, .syncdb, .gho, .cas, .svg, .map, .wmo, .itm, .sb, .fos, .mcgame, .vdf, .ztmp, .sis, .sid, .ncf, .menu, .layout, .dmp, .blob, .esm, .001, .vtf, .dazip, .fpk, .mlx, .kf, .iwd, .vpk, .tor, .psk, .rim, .w3x, .fsh, .ntl, .arch00, .lvl, .snx, .cfr, .ff, .vpp_pc, .lrf, .m2, .mcmeta, .vfs0, .mpqge, .kdb, .db0, .mp3, .upx, .rofl, .hkx, .bar, .upk, .das, .iwi, .litemod, .asset, .forge, .ltx, .bsa, .apk, .re4, .sav, .lbf, .slm, .bik, .epk, .rgss3a, .pak, .big, .unity3d, .wotreplay, .xxx, .desc, .py, .m3u, .flv, .js, .css, .rb, .png, .jpeg, .p7c, .p7b, .p12, .pfx, .pem, .crt, .cer, .der, .x3f, .srw, .pef, .ptx, .r3d, .rw2, .rwl, .raw, .raf, .orf, .nrw, .mrwref, .mef, .erf, .kdc, .dcr, .cr2, .crw, .bay, .sr2, .srf, .arw, .3fr, .dng, .jpeg, .jpg, .cdr, .indd, .ai, .eps, .pdf, .pdd, .psd, .dbfv, .mdf, .wb2, .rtf, .wpd, .dxg, .xf, .dwg, .pst, .accdb, .mdb, .pptm, .pptx, .ppt, .xlk, .xlsb, .xlsm, .xlsx, .xls, .wps, .docm, .docx, .doc, .odb, .odc, .odm, .odp, .ods, .odt, .sql, .zip, .tar, .tar.gz, .tgz, .biz, .ocx, .html, .htm, .3gp, .srt, .cpp, .mid, .mkv, .mov, .asf, .mpeg, .vob, .mpg, .fla, .swf, .wav, .qcow2, .vdi, .vmdk, .vmx, .gpg, .aes, .ARC, .PAQ, .tar.bz2, .tbk, .bak, .djv, .djvu, .bmp, .cgm, .tif, .tiff, .NEF, .cmd, .class, .jar, .java, .asp, .brd, .sch, .dch, .dip, .vbs, .asm, .pas, .ldf, .ibd, .MYI, .MYD, .frm, .dbf, .SQLITEDB, .SQLITE3, .asc, .lay6, .lay, .ms11 (Security copy), .sldm, .sldx, .ppsm, .ppsx, .ppam, .docb, .mml, .sxm, .otg, .slk, .xlw, .xlt, .xlm, .xlc, .dif, .stc, .sxc, .ots, .ods, .hwp, .dotm, .dotx, .docm, .DOT, .max, .xml, .uot, .stw, .sxw, .ott, .csr, .key, wallet.dat

Encrypted files have an extension of .fucku appended to the original filename. This can be seen in the decompiled code as shown below.

Figure 8: Supported extensions

Files are encrypted with the AES cipher in CBC mode with a random IV generated per file.

Figure 9: Files are encrypted using AES

AES is no doubt a strong algorithm and infeasible to crack. However, the ransomware encrypts each file using a constant and hardcoded key which makes decryption feasible. This is shown in the figure below. The AES key used is j<L;G|hD*3CQk%I!g|Ei&#aQ6*;Vh,

Figure 10: Look, the password is hardcoded!

Decrypting encrypted files

Since we know that each files are encrypted with the same key we can develop a decrypter. However, our kind ransomware author has spared us the bother by providing the decrypter in the same code.

Figure 11: Bundled decrypter

The function decrypt decrypts an encrypted file. It's not called from anywhere, indicating it was there for testing purposes and was not removed in the final build.

There is no need to pay the ransom if someone is infected by this ransomware. A free decrypter is available from the malware hunter team. Kudos to them for their fabulous work!

Pyinstaller Extractor updated to v1.9

PyInstaller Extractor has been updated to v1.9. The features of this release includes:

• Support for Pyinstaller 3.3
• Display the scripts which are run at entry point 

Support for Pyinstaller 3.3

Self explanatory. For extending the support to Pyinstaller 3.3 no major changes had to be introduced. The earlier script works as-is.

Display the scripts which are run at entry point

A Pyinstaller executable have many embedded files in it. Naturally, users of this tool had difficulty identifying which of the extracted files are of interest. With this update, pyinstxtractor now shows a list of python scripts which are run by the executable at load time. An example is shown in the screenshot below.

pyiboot01_bootstrap and main are the scripts which are run at load time. Out of this two, the former is Pyinstaller specific and not interesting for our purpose. Hence you should start the analysis from the file named main located within the _extracted directory.

As usual, pyinstxtractor can be found at SourceForge.

TUCTF Write-up - RE track

TU CTF is an introductory CTF for teams that want to build their experience. We will have the standard categories of Web, Forensics, Crypto, RE, and Exploit, as well as some other categories we don't want to reveal just yet. If you have any questions, our contact is at the bottom of each page, but please read the official rules before sending us any emails.

This is a write-up for the Reversing challenges in TU CTF 2017.

Funmail [25]

Figure 1: Challenge description

This is straightforward. The challenge requires a password which is hardcoded within the binary as shown in the Figure 2.

Figure 2: Hardcoded password

bnpy - A python architecture plugin for Binary Ninja

Recently I got a chance to try out Vector 35's Binary Ninja, and I must say the experience has been great so far. The good thing about binary ninja (binja henceforth) is its API, we can easily custom plugins for various purposes such as a disassembler for a foreign architecture. We can do the same in IDA, but developing processor plugins in IDA is not for the faint of heart. At the moment, binja is  entirely a static analysis tool but we do have plugins like binjatron that attempts to fill this void.

Playing with the binja API, I developed bnpy - a disassembler for python bytecode. In the binja terminology this is called as an Architecture plugin. At the moment it works for raw python bytecode, i.e. you must extract the instruction stream from a pyc file in order to use it.

In the near future, I plan to extend it so that it can disassemble a pyc (compiled python) file right out of the box. Right now, this is difficult due to certain limitations in the API. To understand this we need to know a bit more about the pyc format.

The pyc file is not a flat file format like a PE or ELF.  It is a nested format bearing a tree-like structure. A pyc file contains a single top-level code object. Among other things, a code object stores an array of constants used by the code. This array is called as co_consts. The constants can be integers, strings and even another nested code object. The code object also stores the bytecode instructions in a string named as co_code. At the moment, the bnpy plugin operates on this instruction string. To better describe the structure of pyc files we can refer to  the following image taken from kaitai struct.

Fig. 1: The structure of a pyc file

You can see, the code objects within a pyc file are nested. The function view in binja is flat and thus not suitable for displaying a tree structure. As of now, the plugin can be used on the raw bytecode stream. Steps for extracting the bytecode along with other directions can be found on the plugin page at GitHub.

https://github.com/extremecoders-re/bnpy

To conclude this short post, here is a GIF of the plugin in action.

Flare-On Challenge 2017 Writeup

Flare-on is an annual CTF style challenge organized by Fire-eye with a focus on reverse engineering. The contest falls into its fourth year this season. Taking part in these challenges gives us a nice opportunity to learn something new and this year was no exception. Overall, there were 12 challenges to complete. Official solution to the challenges has already been published at the FireEye blog. Hence instead of a detailed write-up, I will just cover the important parts.

#1 - Login.html

The first problem was as simple as it gets. There is an HTML file with a form. We need to provide a flag and check for its correctness.

Figure 1: Check thy flag

The code simply performs a ROT-13 of the input and compares it with another string. To get back the flag, re-apply ROT-13.

Figure 2: ROT-13 again

Flag: ClientSideLoginsAreEasy@flare-on.com

Deobfuscating PjOrion using bytecode simplifier

Bytecode simplifier is a tool to de-obfuscate PjOrion protected python scripts. This post is a short tutorial to show how to use this module to deobfuscate a protected python script.

I have used the sample code below to demonstrate its usage. This is a small program to calculate the factorial of a number.

# Python program to find the factorial of a number using recursion
 
def recur_factorial(n):
   """Function to return the factorial
   of a number using recursion"""
   if n == 1:
       return n
   else:
       return n*recur_factorial(n-1)
 
 
# take input from the user
num = int(input("Enter a number: "))
 
# check is the number is negative
if num < 0:
   print("Sorry, factorial does not exist for negative numbers")
elif num == 0:
   print("The factorial of 0 is 1")
else:
   print("The factorial of",num,"is",recur_factorial(num))

Introducing bytecode simplifier

Bytecode simplifier is a tool to deobfuscate PjOrion protected python scripts. It is a complete rewrite of my earlier tool PjOrion Deobfuscator. I have reimplemented the deobfuscation functionality from scratch and have used networkx specifically for this purpose. Using networkx made reasoning about the code much simpler.
The PjOrion version used is 1.3.2 (Filename: PjOrion_Uncompyle6_01.10.2016.zip)

The code is at https://github.com/extremecoders-re/bytecode_simplifier

A short tutorial can be found here: https://0xec.blogspot.com/2017/07/deobfuscating-pjorion-using-bytecode.html