test_uc_hook_cached_uaf test fails with SIGSEGV after building on x86 with musl

[strophy]

Hi, I'm trying to update the unicorn package on Alpine Linux to v2.1.2. Compilation succeeds but I'm getting persistent test failures running test_uc_hook_cached_uaf (link) on x86 arch as follows:

Test project /builds/strophy/aports/testing/unicorn/src/unicorn-2.1.2/build
      Start  1: test_x86
      Start  2: test_arm
      Start  3: test_arm64
      Start  4: test_m68k
      Start  5: test_mips
      Start  6: test_sparc
      Start  7: test_ppc
      Start  8: test_riscv
      Start  9: test_s390x
      Start 10: test_tricore
      Start 11: test_mem
      Start 12: test_ctl
 1/12 Test  #4: test_m68k ........................   Passed    0.01 sec
 2/12 Test  #6: test_sparc .......................   Passed    0.01 sec
 3/12 Test #10: test_tricore .....................   Passed    0.01 sec
 4/12 Test  #9: test_s390x .......................   Passed    0.01 sec
 5/12 Test  #5: test_mips ........................   Passed    0.02 sec
 6/12 Test  #7: test_ppc .........................   Passed    0.02 sec
 7/12 Test  #3: test_arm64 .......................   Passed    0.05 sec
 8/12 Test #11: test_mem .........................   Passed    0.04 sec
 9/12 Test  #8: test_riscv .......................   Passed    0.04 sec
10/12 Test #12: test_ctl .........................***Failed    0.05 sec
Test test_uc_ctl_mode...                        [ OK ]
Test test_uc_ctl_page_size...                   [ OK ]
Test test_uc_ctl_arch...                        [ OK ]
Test test_uc_ctl_time_out...                    [ OK ]
Test test_uc_ctl_exits...                       [ OK ]
Test test_uc_ctl_tb_cache...                    [ OK ]
Test test_uc_ctl_change_page_size...            [ OK ]
Test test_uc_ctl_arm_cpu...                     [ OK ]
Test test_uc_ctl_change_page_size_arm64...      [ OK ]
Test test_uc_hook_cached_uaf...                   Test interrupted by SIGSEGV.
Test test_uc_emu_stop_set_ip...                 [ OK ]
Test test_tlb_clear...                          [ OK ]
Test test_noexec...                             [ OK ]
FAILED: 1 of 13 unit tests has failed.
11/12 Test  #2: test_arm .........................   Passed    0.15 sec
12/12 Test  #1: test_x86 .........................   Passed    1.11 sec
92% tests passed, 1 tests failed out of 12
Total Test time (real) =   1.12 sec
The following tests FAILED:
	 12 - test_ctl (Failed)
Errors while running CTest

I'm a devops engineer and don't know anything about C or assembly, so I worked on it for a while with Copilot to try and narrow down where the test is failing. It helped me modify the function to add debug prints, and I managed to get it the test to pass by using a static callback function and not using the memcpy() call. Obviously this defeats the purpose of the test, so I added back the previous steps with debug logs as follows:

static void simple_callback(uc_engine *uc, uint64_t addr, size_t size, void *user_data)
{
    printf("Callback triggered\n");
    fflush(stdout);
}

static void test_uc_hook_cached_uaf(void)
{
    printf("Starting test_uc_hook_cached_uaf\n");
    fflush(stdout);
    uc_engine *uc;
    // "INC ecx; DEC edx; jmp t; t: nop"
    char code[] = "\x41\x4a\xeb\x00\x90";
    uc_hook h;
    uint64_t count = 0;

    // Step 1: Allocate aligned memory for the callback function
#ifndef _WIN32
    void *callback;
    if (posix_memalign(&callback, 4096, 4096) != 0) {
        perror("posix_memalign failed");
        return;
    }
    if (mprotect(callback, 4096, PROT_READ | PROT_WRITE) != 0) {
        perror("mprotect failed");
        free(callback);
        return;
    }
#else
    void *callback = VirtualAlloc(NULL, 4096, MEM_RESERVE | MEM_COMMIT,
                                  PAGE_EXECUTE_READWRITE);
    if (callback == NULL) {
        fprintf(stderr, "VirtualAlloc failed\n");
        return;
    }
#endif

    printf("Callback memory allocated\n");
    fflush(stdout);

    // Step 2: Copy the callback function to the allocated memory
    memcpy(callback, (void *)simple_callback, 4096);

    printf("Callback memory copied\n");
    fflush(stdout);

    // Step 3: Set memory protection to read and execute
#ifndef _WIN32
    if (mprotect(callback, 4096, PROT_READ | PROT_EXEC) != 0) {
        perror("mprotect failed");
        free(callback);
        return;
    }
#endif

    printf("Callback memory protected\n");
    fflush(stdout);

    uc_common_setup(&uc, UC_ARCH_X86, UC_MODE_32, code, sizeof(code) - 1);

    OK(uc_hook_add(uc, &h, UC_HOOK_CODE, (void *)callback, &count, 1, 0));
    printf("Hook added\n");
    fflush(stdout);

    OK(uc_emu_start(uc, code_start, code_start + sizeof(code) - 1, 0, 0));
    printf("First emulation start\n");
    fflush(stdout);

    // Move the hook to the deleted hooks list.
    OK(uc_hook_del(uc, h));
    printf("Hook deleted\n");
    fflush(stdout);

    // This will clear deleted hooks and SHOULD clear cache.
    OK(uc_emu_start(uc, code_start, code_start + sizeof(code) - 1, 0, 0));
    printf("Second emulation start\n");
    fflush(stdout);

    memset(callback, 0, 4096);

    // Now hooks are deleted and thus this will trigger a UAF
    OK(uc_emu_start(uc, code_start, code_start + sizeof(code) - 1, 0, 0));
    printf("Third emulation start\n");
    fflush(stdout);

    TEST_CHECK(count == 4);

    OK(uc_close(uc));

#ifndef _WIN32
    free(callback);
#else
    VirtualFree(callback, 0, MEM_RELEASE);
#endif
    printf("Test completed\n");
    fflush(stdout);
}

This results in the following test failure during build:

10/12 Test #12: test_ctl .........................***Failed    0.04 sec
Test test_uc_ctl_mode...                        [ OK ]
Test test_uc_ctl_page_size...                   [ OK ]
Test test_uc_ctl_arch...                        [ OK ]
Test test_uc_ctl_time_out...                    [ OK ]
Test test_uc_ctl_exits...                       [ OK ]
Test test_uc_ctl_tb_cache...                    [ OK ]
Test test_uc_ctl_change_page_size...            [ OK ]
Test test_uc_ctl_arm_cpu...                     [ OK ]
Test test_uc_ctl_change_page_size_arm64...      [ OK ]
Test test_uc_hook_cached_uaf...                 Starting test_uc_hook_cached_uaf
Callback memory allocated
Callback memory copied
Callback memory protected
Hook added
  Test interrupted by SIGSEGV.
Test test_uc_emu_stop_set_ip...                 [ OK ]
Test test_tlb_clear...                          [ OK ]
Test test_noexec...                             [ OK ]
FAILED: 1 of 13 unit tests has failed.

I'm not familiar with tools like gdb or valgrind and don't have direct access to x86 hardware so I thought this might be a good time to stop and ask for help. Can anyone help me get past this issue? Is it related to the emulator, or could it be something to do with musl?

You can see my build attempts and log output here: https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/80178

[wtdcode]

What's your build script? Is it possible to reproduce within a alpine container?

[strophy]

Thanks for the quick response, I was able to reproduce this locally in a linux/386 Alpine container as well. Here are the steps (you'll need to modify the suffix on the keys below):

1. Generate build keys:

docker run --name keys --entrypoint abuild-keygen --env PACKAGER="test <[email protected]>" strophy/alpine-abuild:3.21 -n
docker cp keys:/home/builder/.abuild/[email protected] ~/.abuild/
docker cp keys:/home/builder/.abuild/[email protected] ~/.abuild/
docker rm -f keys

2. Clone my branch: git clone -b unicorn-2.1.2 https://gitlab.alpinelinux.org/strophy/aports.git
3. Run the build:

cd aports/testing/unicorn
docker run \
  -e RSA_PRIVATE_KEY="$(cat ~/.abuild/[email protected])" \
  -e RSA_PRIVATE_KEY_NAME="[email protected]" \
  -v "$PWD:/home/builder/package" \
  -v "$HOME/.abuild/packages:/packages" \
  -v "$HOME/.abuild/[email protected]:/etc/apk/keys/[email protected]" \
  -v "$HOME/.abuild:/home/builder/.abuild" \
  -it --ulimit nofile=262144:262144 \
  --platform linux/386 \
  strophy/alpine-abuild:3.21

It should be possible to step inside the container for debugging by modifying the APKBUILD file to call bash just before the check runs like this:

check() {
	cd build
	bash
	CTEST_OUTPUT_ON_FAILURE=TRUE ctest
}

[wtdcode]

Link to #2108

[strophy]

Thanks for checking this. Note that we apply this patch before running tests: https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/80178/diffs

@Antelox you might need this as well to get the CI builds working?

[wtdcode]

Thanks for checking this. Note that we apply this patch before running tests: https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/80178/diffs

@Antelox you might need this as well to get the CI builds working?

What's the rationale of changing the endian macros?

[strophy]

I'm not the original maintainer, but my guess is because the initial condition of #if defined(__GLIBC__) is not defined on musl systems, but we still want to use the logic from that block as if that condition is satisfied, because musl still provides endian.h and defines __BYTE_ORDER in the same way as a GNU system. Does that sound right, am I understanding the patch correctly?

The other part of the patch modifying qemu/target/ppc/mem_helper.c is to resolve the following compile-time error:

/home/builder/package/src/unicorn-2.1.2/qemu/target/ppc/mem_helper.c: In function 'helper_stq_le_parallel':
/home/builder/package/src/unicorn-2.1.2/qemu/target/ppc/mem_helper.c:423:12: error: 'return' with a value, in function returning void [-Wreturn-mismatch]
  423 |     return 0;
      |            ^
/home/builder/package/src/unicorn-2.1.2/qemu/target/ppc/mem_helper.c:411:6: note: declared here
  411 | void helper_stq_le_parallel(CPUPPCState *env, target_ulong addr,
      |      ^~~~~~~~~~~~~~~~~~~~~~
/home/builder/package/src/unicorn-2.1.2/qemu/target/ppc/mem_helper.c: In function 'helper_stq_be_parallel':
/home/builder/package/src/unicorn-2.1.2/qemu/target/ppc/mem_helper.c:439:12: error: 'return' with a value, in function returning void [-Wreturn-mismatch]
  439 |     return 0;
      |            ^
/home/builder/package/src/unicorn-2.1.2/qemu/target/ppc/mem_helper.c:427:6: note: declared here
  427 | void helper_stq_be_parallel(CPUPPCState *env, target_ulong addr,
      |      ^~~~~~~~~~~~~~~~~~~~~~