Challenge 3 - Uncovering Multiple layers of Hidden Data

In the previous challenge, we recovered a hidden document from a JPEG image using Steghide and StegSeek. Challenge 3 takes a different approach. Instead of relying on a single hidden file, we need to investigate a JPEG containing several embedded data structures, carve those structures manually, and follow multiple layers of encryption and steganography to recover the flag.

This walkthrough covers the investigation from the initial file checks to the final archive extraction. We will use tools such as file, ExifTool, Binwalk, xxd, dd, OpenSSL, and a Python extraction script supplied with the challenge.

Lab note: The filenames, offsets, and password pattern in this article belong to this challenge. Your results may differ when analysing another file. Run these steps only against files and systems you own or are authorised to investigate.

Understanding the Challenge

Prepare the Challenge Files

Move into the directory for Challenge 3 in the downloaded challenge repository. The exact path depends on where you saved the repository. In this example, we are working from the challenge directory and using Honey.jpeg as the target image.

List the available files:

The directory contains two important files: the challenge description and the target JPEG image. 

Read the description before proceeding. It may contain clues that become useful later in the investigation.

For the initial examination, we will focus on Honey.jpeg. We can return to the description when we need to interpret the challenge-specific clues.

Inspect the Image Format

A file extension does not prove a file's actual format. A text document or encrypted file can be renamed with a `.jpg` extension without becoming a JPEG image. We should inspect the file contents before trusting the extension.

Use the `file` command:

The challenge file is identified as JPEG image data using the EXIF standard, with dimensions of 1024 × 1024 pixels. This confirms that the file contains a JPEG image structure.

However, a valid JPEG can still contain additional data. The next step is to inspect its metadata.

Inspect the JPEG Metadata

Image metadata can include information such as the camera model, image dimensions, orientation, date and time, colour space, and—in some cases—location details. Metadata can also contain clues deliberately added to a CTF challenge.

Use ExifTool to inspect the image:

The output displays the image's technical properties, including its dimensions, EXIF version, orientation, colour space, and JPEG encoding information.

In this challenge, the metadata does not reveal an obvious hidden message. In particular, there is no useful encrypted message in the EXIF Comment field.

That does not mean the image is clean. Metadata is only one place where information can be stored. Additional content may be appended to the file or embedded between its normal structures. To investigate the complete binary file, we need a signature-scanning tool.

Scan the Image with Binwalk

Binwalk searches files for recognisable signatures that may indicate embedded files or data structures. For example, a JPEG may contain a ZIP archive, another image, or an encrypted data section.

Run a standard Binwalk scan:

The output contains three columns:

• Decimal: The byte offset where Binwalk identifies a structure, shown in decimal notation.
• Hexadecimal: The same offset represented in hexadecimal notation.
• Description: The type of structure or file signature detected.

The important findings in this challenge are:

The entries at offsets 235,921, 272,710, and 272,950 are especially interesting. They indicate three sections that appear to use OpenSSL's salted encryption format. Binwalk also identifies a ZIP archive and a second JPEG image.

These offsets mark the detected starting positions. They do not, by themselves, tell us the exact length of every structure. We must examine the boundaries before extracting each section.

Try Binwalk's Automatic Extraction

Binwalk can attempt to extract recognised embedded files. The -e option enables extraction, while -M enables recursive scanning, sometimes called Matryoshka extraction. With recursive extraction, Binwalk scans extracted files again for additional embedded structures.

Run:

Binwalk creates an extraction directory, usually named: _Honey.jpeg.extracted/

The output also reports that one or more items could not be extracted. This does not mean the signature scan was wrong. Identifying a structure and extracting or decrypting it are separate operations.

Binwalk may recognise an OpenSSL header but cannot determine the password needed to decrypt the data. It may also lack the extraction utility or rule required for a particular structure.

Inspect the files that were extracted. Change the directory and list all files and directories.

In this challenge, the directory contains:

• 3C3D6.zip
• do_not_open.txt

Inspect do_not_open.txt

Now, we have to inspect the text file to get any clues:

Its contents may look like a clue, but we should not assume that it is relevant. The directory also contains a ZIP archive, so inspect the archive before spending time on the text file.

Inspect the extracted ZIP archive

List the archive contents without extracting them:

The archive contains another file named do_not_open.txt. It is the same decoy file already found in the extraction directory. This branch does not lead to the main solution, so we can set it aside and return to the structures reported by the original Binwalk scan.

Return to the challenge directory:

Prepare for Manual File Carving

Binwalk identified multiple structures, but automatic extraction did not recover everything. We will therefore extract the required byte ranges ourselves. This process is known as file carving.

File carving is a digital forensics technique used to recover data from a larger file by identifying a structure's starting position and ending boundary. We can then copy only that range into a separate output file.

For this task, we will use two Linux utilities:

• `xxd` displays file contents in hexadecimal form. It is useful for checking file signatures and inspecting bytes at specific offsets.
• `dd` copies selected data from an input file to an output file. By setting the starting offset and byte count, we can carve an individual structure from the original image.

Create a directory for the recovered sections:

Check the original file size

We need the original file size to calculate the length of the final section, which continues to the end of `Honey.jpeg`. Use `stat`:

The file size is: 277078

Keep this value. We will use it when carving the final encrypted section.

Understand the `xxd` options

The general form used in this walkthrough is:

Here:

• `-s` sets the starting offset.
• `-l` sets the number of bytes to display.

For example, `xxd -s 235921 -l 16 Honey.jpeg` displays 16 bytes beginning at decimal offset 235,921.

Understand the `dd` options

The general form for our carving commands is:

The options mean:

• `if` specifies the input file.
• `of` specifies the output file.
• `bs=1` sets the block size to one byte, allowing precise byte-based offsets.
• `skip` tells `dd` how many bytes to skip from the beginning of the input file.
• `count` specifies how many bytes to copy after the skipped bytes.
• `status=progress` displays the copy progress.

To calculate the number of bytes to carve, subtract the starting offset from the ending boundary:

Length = Ending boundary - Starting offset

For example, a section beginning at offset 100 and ending at boundary 200 has a length of 100 bytes. The ending boundary is exclusive: the byte at that boundary belongs to the next section and is not included in the current extraction.

Do not assume that the next Binwalk result always marks the exact end of the current structure.

There may be padding, unrelated bytes, comments, or other data between structures. Inspect the actual signatures and boundaries before choosing a byte count.

Carve the First OpenSSL-Encrypted Section

Binwalk reports the first encrypted section at decimal offset 235,921. Before extracting it, inspect its first 16 bytes:

The output begins with: _salted.

This marker is commonly found in OpenSSL's password-based encryption file format. It is followed by salt information used by the key-derivation process.

Calculate the section length

The next ZIP archive begins at offset 246,742. However, the boundary established during this investigation for the encrypted section is 246,721, leaving a 21-byte gap before the ZIP archive begins.

Use the verified ending boundary rather than assuming the encrypted data continues directly to the ZIP signature:

• Ending boundary: 246721
• Starting offset: 235921
• Length: 246721 - 235921 = 10800 bytes

The length is 10,800 bytes. It is divisible by 16, which is consistent with the block alignment expected for an AES-CBC encrypted payload, including the 16-byte OpenSSL salted header. This is a useful consistency check, although block alignment alone does not prove the encryption algorithm.

Extract the encrypted section

Run:

Verify the recovered file:

The extracted file should retain the OpenSSL `Salted__` header.

Carve the ZIP Archive

The ZIP archive begins at decimal offset 246,742. To determine its ending boundary, Binwalk's output identifies a ZIP end-of-central-directory record at offset 246,957.

First, inspect the beginning of the archive:

The bytes identify the start of a ZIP local file header. Next, inspect the end-of-central-directory record:

The record begins with the ZIP end-of-central-directory signature. In this challenge, the record is 22 bytes long and has no additional ZIP comment. Therefore:

246957 + 22 = 246979

The calculated ending boundary, 246,979, matches the starting offset of the next JPEG image reported by Binwalk. 

This confirms the boundary for this archive. In other ZIP files, the end-of-central-directory record may include a comment, so its comment-length field must also be checked rather than assuming the record is always exactly 22 bytes.

Calculate the ZIP archive's length:

• Ending boundary: 246979
• Starting offset: 246742
• Length: 246979 - 246742 = 237 bytes

Extract the archive:

Verify and test it:

The extracted file should be identified as a ZIP archive, and the integrity test should complete successfully. Extract it into a separate directory if you want to inspect its contents:

The archive contains `do_not_open.txt`, the same decoy file discovered during Binwalk's automatic extraction. We have now confirmed that this ZIP archive is not the main route to the flag.

Carve the Second JPEG Image

The next embedded structure begins at offset 246,979 and is another JPEG image. 

Check its starting bytes:

A JPEG image normally begins with the start-of-image marker FF D8. The output confirms that marker at the expected location.

Find the JPEG ending boundary

The next encrypted structure begins at offset 272,710. To check the bytes immediately before that offset, inspect the preceding 16 bytes:

The output includes the JPEG end-of-image marker, FF D9, at the end of the range. This indicates that the JPEG ends immediately before offset 272,710.

Calculate the image length:

• Ending boundary: 272710
• Starting offset: 246979
• Length: 272710 - 246979 = 25731 bytes

Extract and inspect the image:

Carve the image into a separate file:

Verify the recovered file and inspect its metadata:

Also, we use exiftool to understand:

The extracted file is a JPEG image. Its metadata does not reveal an obvious clue, so we will retain it for later analysis. It becomes important when we investigate the JPEG quantization tables.

Carve the Fourth Structure: Another Encrypted Section

Binwalk identifies another OpenSSL-encrypted structure at offset 272,710. 

Inspect its first 16 bytes:

Once again, the data begins with the `Salted__` marker.

The next encrypted section begins at offset 272,950. For this extraction, the established boundary of the current section is the start of that next structure.

Calculate the length:

• Ending boundary: 272950
• Starting offset: 272710
• Length: 272950 - 272710 = 240 bytes

The 240-byte length is divisible by 16. This is consistent with the block alignment expected for an OpenSSL salted AES-CBC payload, but it is not, by itself, conclusive proof of the cipher.

Extract the section:

Verify the result:

The extracted data should begin with the same OpenSSL salted header.

Carve the Fifth and Final Structure

The final encrypted section begins at offset 272,950. 

First, confirm its starting bytes:

The `Salted__` header appears again.

Unlike the previous sections, there is no later embedded structure that marks the ending boundary. This section continues to the end of the original file. We therefore use the file size obtained earlier: 277,078 bytes.

Calculate the number of bytes remaining:

• File size: 277078
• Starting offset: 272950
• Length: 277078 - 272950 = 4128 bytes

Extract the final section:

Verify the file and its header:

The output should be 4,128 bytes long and should begin with the OpenSSL salted marker. The earlier JPEG metadata inspection did not reveal a useful clue, so we can leave that image aside for now.

We have now recovered all five structures identified during the initial scan.

The table lists the byte ranges used in this walkthrough. The ending boundary is exclusive, meaning the byte at the ending offset is not included in the extracted file.

At this stage, we have three encrypted files, one ZIP archive, and one additional JPEG image. The ZIP archive is a decoy. The next step is to decrypt the first OpenSSL file and examine the material inside it.

Decrypt the secret.enc

From our earlier investigation, we already know that this file contains OpenSSL-encrypted data with a salted header. This means we cannot simply open it and read its original contents.

We need the correct decryption settings and, most importantly, the password.

So, where do we find the password?

This is where the challenge description becomes useful. Remember, the challenge brief contains some clues about the password format.

Let's examine those clues carefully.

The challenge description contains a password pattern. It specifies a codename followed by a hash symbol and five characters with particular character types:

For this challenge:

• `?d` represents a digit.
• `?l` represents a lowercase letter.
• `?u` represents an uppercase letter.

The codename is Desert Storm. The pattern requires two digits, then a lowercase letter, an uppercase letter, and another lowercase letter. Applying the challenge's intended formatting gives this password:

The password omits the space in the displayed codename. 

Now, we've got a possible password that matches the pattern provided in the challenge. Let's use it to decrypt the first encrypted file.

Decrypt the OpenSSL File

The first carved encrypted file is `carved/secret.enc`. Its `Salted__` header indicates that we should use OpenSSL's password-based decryption workflow.

Run:

The options are:

• `enc` selects the symmetric encryption command.
• `-d` enables decryption.
• `-aes-256-cbc` selects AES-256 in CBC mode.
• `-pbkdf2` enables PBKDF2 password-based key derivation.
• `-pass` supplies the password.
• `-in` specifies the encrypted input file.
• `-out` specifies the decrypted output file.

Identify the recovered file:

The output identifies it as a ZIP archive. We have successfully decrypted the first encrypted section, but we still need to inspect the archive contents.

Test and extract the archive

Test the archive without extracting it:

The archive passes the integrity test. Extract it into a separate directory:

List the recovered files:

The directory contains four files:

• iv.bin
• passwords.enc
• qtbl.py
• STEGO_KEY_368.txt

The filenames provide useful leads:

• `iv.bin` appears to contain an initialization vector.
• `passwords.enc` contains encrypted password data.
• `qtbl.py` is a Python script that extracts hidden information from JPEG quantization tables.
• `STEGO_KEY_368.txt` contains data that appears to be used to construct the script's extraction key.

These filenames suggest that the files are related, but we shouldn't assume their purpose just from their names. We will examine these files rather than assuming their purpose from their names alone. 

Inspect the Encrypted Password File

Start by identifying the format of `passwords.enc`:

The `file` utility does not recognise a familiar format. That does not necessarily indicate corruption. The file may contain raw encrypted bytes without a standard file header.

Inspect its first 32 bytes:

The output looks like ciphertext, not readable text. Unlike `secret.enc`, it does not begin with the OpenSSL `Salted__` marker.

This suggests that `passwords.enc` may use raw encryption data with the key and initialization vector supplied separately. 

What does this mean? It means that when we have the correct AES key and initialization vector, or IV, we can use OpenSSL to decrypt the encrypted file, provided we also specify the correct encryption algorithm and mode.

Finding and Verifying the AES Key and IV Before Decryption

AES encryption requires a key. In CBC mode, it also uses an initialization vector, commonly abbreviated as IV.

The key is the secret value used to encrypt and decrypt the data. The IV is an additional input used by CBC mode. For AES-CBC, the IV must be 16 bytes long, regardless of whether the AES key is 128, 192, or 256 bits.

Here, in this directory, we have an IV file. Inspect the IV file:

The displayed hexadecimal data contains 32 hexadecimal characters. Each pair of hexadecimal characters represents one byte, so the file contains 16 bytes.

Confirm the exact size:

The result is: 16 secret_bundle/iv.bin

This matches the required IV size for AES-CBC. However, the IV is not the AES key. We still need to recover the key before we can decrypt `passwords.enc`.

Understand the Python script

Before examining the key file, inspect the supplied Python script:

You can also display it directly with `cat`, although `less` is usually easier for longer scripts. Press `q` to exit `less`.

The script reads a JPEG image and searches for its DQT, or Define Quantization Table, data. Quantization tables are part of JPEG compression. The script examines selected values in those tables and extracts their two least significant bits to reconstruct hidden bytes.

This is a form of steganography: information is concealed in values that are part of the JPEG structure, rather than being stored as a normal visible message.

The script also accepts an optional key. According to its implementation, the supplied value is used in an XOR operation on the extracted bytes to reverse an additional layer of obfuscation. The recovered bytes are then written to an output file.

In other words, `qtbl.py` is not simply reading ordinary image metadata. It is designed to recover data hidden inside the JPEG quantization tables. 

Now, let's see how we're supposed to use it.

Read the script's usage information

Run the script's help command:

As you can see, it displays its help information.

Here, there is only 1 option, which is --help, not good for the task. So, let me take a look at the help option for extract.

The extraction operation accepts an input image, an output filename, and a key. The relevant options are:

• `-i` specifies the input image.
• `-o` specifies the output file.
• `-k` supplies the extraction key.

Fortunately, we already have the image recovered from our previous steps. It's named `carved/nothingtoseehere.jpg`. 

But what about the key? While looking to get hint or clue to get the key, I  remembered a interesting file, here, we need to investigate the file which is inside our extracted directory.

It's named STEGO_KEY_368.txt.

Construct the Extraction Key from `STEGO_KEY_368.txt`

The file `STEGO_KEY_368.txt` is the next clue. Its name contains the numbers 3, 6, and 8, which may indicate which records we need to select.

First, inspect the file:

The contents appear as one long, difficult-to-read string. Press `q` to exit.

Check the number of lines and bytes

Count the newline characters with:

The result is zero. This does not mean the file is empty. It means the file contains no newline characters separating its contents into multiple lines.

Now check the total size in bytes:

The file contains 4,824 bytes. The challenge organises the data into records of 24 characters each. Divide the file size by the record width:

4824 / 24 = 201 records

The file therefore contains 201 records of 24 characters each, stored as one continuous string.

Split the string into 24-character records

The Linux `fold` utility can wrap a long line at a specified width. Use `-w 24` to divide the data into 24-character records.

Here, this command performs three operations:

• `fold -w 24` wraps the continuous string into records that are 24 characters wide.
•  `nl` numbers the resulting records.
•  `head` displays the first ten records by default.

The output makes the record positions easier to inspect. So, this means all the characters are stored on a single line. We need to split this long string into smaller records first.

The challenge filename, `STEGO_KEY_368.txt`, indicates that we need to select records three, six, and eight, in that exact order, and combine them into a single extraction key.

So, how do we do that? Remember the clue from the challenge brief? It mentions AWK.

But what exactly is AWK? 

Select records 3, 6, and 8

AWK is a text-processing language commonly used on Linux. It can select specific records and process their contents. We will use it to extract records 3, 6, and 8, then join them without adding spaces or line breaks.

Run:

The command works as follows:

• `fold -w 24` splits the original string into 24-character records.
• `awk` checks the record number stored in `NR`.
• The condition selects records 3, 6, and 8. 
• `printf "%s", $0` prints each selected record without adding a newline.
• `KEY=$(...)` stores the combined output in a shell variable named `KEY`.

Check the key length:

The result is 72. That is expected because we selected three records, each containing 24 characters:

3 × 24 = 72 characters

Display the resulting value if you need to inspect it:

This 72-character string is the extraction key for `qtbl.py`. It isn't the final AES key. The Python script uses it to recover the AES key hidden inside the JPEG image.

Extract the AES Key from the JPEG Image

We now have the two required inputs:

• The recovered JPEG image: `carved/nothingtoseehere.jpg`
• The 72-character extraction key stored in `$KEY`

Run the extraction operation:

The script processes the JPEG quantization tables and writes the recovered bytes to `aeskey.bin`. At this point, we have the AES key file and the IV file. Before attempting decryption, verify their lengths:

The expected sizes are:

• `aeskey.bin`: 32 bytes, which represents a 256-bit AES key.
• `secret_bundle/iv.bin`: 16 bytes, which represents a 128-bit IV.

You can also inspect both files in hexadecimal form:

The key and IV are binary data. OpenSSL's `-K` and `-iv` options expect hexadecimal strings, so we must convert the contents before using them.

Convert the AES Key and IV to Hexadecimal

Use `xxd -p` to print each binary file as a plain hexadecimal string. The `tr` command removes newline characters, and command substitution stores the result in a shell variable.

Create the key variable:

Create the IV variable:

Check the length of the hexadecimal key:

The result should be 64 hexadecimal characters. Each hexadecimal character represents four bits, so 64 characters represent 256 bits, or 32 bytes. That is the expected size of an AES-256 key.

Now check the IV:

The result should be 32 hexadecimal characters, representing 128 bits, or 16 bytes. That is the expected size of an AES-CBC initialization vector.

These length checks help catch common errors, such as extracting an incomplete key or accidentally including extra characters.

Decrypt `passwords.enc`

We now have the AES key and IV required to decrypt `secret_bundle/passwords.enc`.

Because we are supplying the raw key and IV directly, we do not need password-based key derivation with `-pbkdf2`. Use OpenSSL's `-K` option for the hexadecimal key and `-iv` for the hexadecimal initialization vector:

Identify the output:

But what exactly did we recover? Let's identify the output using the file command.

The file is identified as ASCII text with very long lines. 

While inspecting the first six lines, one particular entry caught my eye. It looks interesting and may contain an important clue.

Instead, I'll use the head command with the -n option to display only the first six lines.

The first entries show a filename and its corresponding password. The first pair refers to `secret.enc`, which we have already decrypted. 

The remaining entries provide passwords for other encrypted files from the challenge.

One of the entries is associated with `decoy_random.enc`. Decrypting that file produces data that does not immediately identify itself as a familiar file format. We can inspect it, but it does not reveal a useful route to the flag.

For example, after decrypting the decoy with its corresponding password, inspect the result with:

Let's use xxd:

In this challenge, the output is reported only as generic data. The initial bytes do not show a familiar file signature, and Binwalk does not identify another embedded structure. We can set this result aside and continue with the password entry associated with `payload.enc`.

Extract the Password for `payload.enc`

The sixth line of `passwords.txt` contains the password associated with `payload.enc`. Instead of typing a long password manually, save that line to a separate file.

Use `sed`:

Here:

• `-n` suppresses the normal output of `sed`.
• `6p` prints only line 6.
• `>` redirects the selected line into `pw.txt`.

Check the saved value:

The `$` displayed at the end of the line represents the newline character. It is not part of the password. When OpenSSL reads the password from a file, it handles the line ending as expected.

Using a password file also avoids several shell interpretation problems that can occur when a password contains characters such as `$`, `&`, semicolons, or backticks.

Decrypt `payload.enc`

We now have the encrypted file and its corresponding password. Use OpenSSL with AES-256-CBC and PBKDF2, and tell it to read the password from `pw.txt`:

Identify the decrypted output:

This time, the file utility identifies the output as a ZIP archive. This is a promising result: the decrypted payload contains another archive that may hold the flag.

Test and extract the final archive

Before extracting it, test the ZIP archive's integrity:

The test completes without reporting an error. Extract the archive into a separate directory:

List the extracted files:

Locate the flag file in the output and display it. Use the actual filename shown by your extraction results:

The flag confirms that we have successfully completed Challenge 3.

What We Learned from Challenge 3

This challenge required more than finding a hidden file inside an image. We had to combine several forensic and steganographic techniques:

• File identification: We used `file` to confirm the target's actual format instead of relying on its extension.
• Metadata inspection: We used ExifTool to examine the JPEG's metadata.
• Signature analysis: We used Binwalk to identify embedded encrypted sections, a ZIP archive, and another JPEG image.
• Manual file carving: We used `xxd` to verify boundaries and `dd` to extract exact byte ranges from the original image.
• Password-based decryption: We used a password pattern from the challenge description to decrypt the first OpenSSL-encrypted section.
• JPEG steganography: We examined the supplied Python script and used it to recover an AES key from the image's quantization tables.
• Text processing: We used `fold` and AWK to construct the extraction key from selected records in `STEGO_KEY_368.txt`.
• Raw-key decryption: We converted the recovered binary AES key and IV into hexadecimal strings and used them to decrypt `passwords.enc`.
• Archive analysis: We decrypted `payload.enc`, tested the resulting ZIP archive, and extracted the flag.

The central lesson is to work from evidence. A signature can tell us where a structure begins, but we must verify its boundaries before carving it. A file that the system reports as generic data is not necessarily useless; it may be raw ciphertext or a format without a recognisable header. Finally, encrypted data may require several separate clues—a password pattern, an embedded key, and an initialization vector—before it can be recovered.

That completes the walkthrough for Steganography CTF Challenge 3. Share your questions or observations in the comments, especially if your offsets or extraction results differ from those shown here.