Problem with rsa_parse_pub_key symbol: not found in current Kernel 7.4.7

[PeterGabaldon]

Hello Randori,

First of all thanks for you tool and work publishing your analysis about Fortigate firmware analysis.

I was trying to analyze latest FortiGate firmware 7.4.7 and I have encountered that the tool is not working and found the root cause. I would like to share it with you in order to detect why.

First of all, I downloaded the following FortigateVM image.

I then converted flatkc to ELF for analysis using vmlinux-to-elf. Apparently, the Kernel image has not been modified because it dates to January 20.

~/fgate/vmlinux-to-elf$ ./vmlinux-to-elf ../flatkc ../flatkc.elf                                                                             
[+] Kernel successfully decompressed in-memory (the offsets that follow will be given relative to the decompressed binary)                                  
[+] Version string: Linux version 4.19.13 (root@build) (gcc version 10.3.0 (GCC)) #1 SMP Mon Jan 20 18:15:22 America 2025
[+] Guessed architecture: x86_64 successfully in 2.36 seconds                                                                                               
[+] Found kallsyms_token_table at file offset 0x013d14b8                                                                                                    
[+] Found kallsyms_token_index at file offset 0x013d17f8                                                                                                    
[+] Found kallsyms_markers at file offset 0x013d0f50                                                                                                        
[+] Found kallsyms_names at file offset 0x0134bf78                                                                                                          
[+] Found kallsyms_num_syms at file offset 0x0134bf70                                                                                                       
[i] Null addresses overall: 25.3565 %                                                                                                                       
[+] Found kallsyms_addresses at file offset 0x012f5ae0                                                                                                      
[+] Successfully wrote the new ELF kernel to ../flatkc.elf

After that the tool is not working and I have found that the symbol rsa_parse_pub_key is not available.

Moreover, found that rsa_set_pub_key is not calling rsa_parse_pub_key . According to source code it should (https://github.com/torvalds/linux/blob/v4.19/crypto/rsa.c#L267) but that is not the case.

~/fgate$ objdump -M intel -d --disassemble=rsa_set_pub_key flatkc.elf

flatkc.elf:     file format elf64-x86-64


Disassembly of section .text:

ffffffff80c02cd6 <rsa_set_pub_key>:
ffffffff80c02cd6:       55                      push   rbp
ffffffff80c02cd7:       48 89 e5                mov    rbp,rsp
ffffffff80c02cda:       41 54                   push   r12
ffffffff80c02cdc:       53                      push   rbx
ffffffff80c02cdd:       48 83 ec 50             sub    rsp,0x50
ffffffff80c02ce1:       4c 89 c3                mov    rbx,r8
ffffffff80c02ce4:       65 48 8b 04 25 28 00    mov    rax,QWORD PTR gs:0x28
ffffffff80c02ceb:       00 00
ffffffff80c02ced:       48 89 45 e8             mov    QWORD PTR [rbp-0x18],rax
ffffffff80c02cf1:       48 8b 47 08             mov    rax,QWORD PTR [rdi+0x8]
ffffffff80c02cf5:       8b 00                   mov    eax,DWORD PTR [rax]
ffffffff80c02cf7:       89 45 a4                mov    DWORD PTR [rbp-0x5c],eax
ffffffff80c02cfa:       4d 8b 20                mov    r12,QWORD PTR [r8]
ffffffff80c02cfd:       48 8b 07                mov    rax,QWORD PTR [rdi]
ffffffff80c02d00:       48 89 45 a8             mov    QWORD PTR [rbp-0x58],rax
ffffffff80c02d04:       48 8b 47 10             mov    rax,QWORD PTR [rdi+0x10]
ffffffff80c02d08:       48 89 45 b8             mov    QWORD PTR [rbp-0x48],rax
ffffffff80c02d0c:       48 8b 47 18             mov    rax,QWORD PTR [rdi+0x18]
ffffffff80c02d10:       48 89 45 c0             mov    QWORD PTR [rbp-0x40],rax
ffffffff80c02d14:       48 8b 47 20             mov    rax,QWORD PTR [rdi+0x20]
ffffffff80c02d18:       48 89 45 c8             mov    QWORD PTR [rbp-0x38],rax
ffffffff80c02d1c:       48 8b 47 28             mov    rax,QWORD PTR [rdi+0x28]
ffffffff80c02d20:       48 89 45 d0             mov    QWORD PTR [rbp-0x30],rax
ffffffff80c02d24:       48 8b 47 30             mov    rax,QWORD PTR [rdi+0x30]
ffffffff80c02d28:       48 89 45 d8             mov    QWORD PTR [rbp-0x28],rax
ffffffff80c02d2c:       48 8b 47 38             mov    rax,QWORD PTR [rdi+0x38]
ffffffff80c02d30:       48 89 45 e0             mov    QWORD PTR [rbp-0x20],rax
ffffffff80c02d34:       48 8d 45 a4             lea    rax,[rbp-0x5c]
ffffffff80c02d38:       48 89 45 b0             mov    QWORD PTR [rbp-0x50],rax
ffffffff80c02d3c:       48 8d 7d a8             lea    rdi,[rbp-0x58]
ffffffff80c02d40:       e8 4d 50 6f ff          call   ffffffff802f7d92 <crypto_register_acomps>
ffffffff80c02d45:       85 c0                   test   eax,eax
ffffffff80c02d47:       74 03                   je     ffffffff80c02d4c <rsa_set_pub_key+0x76>
ffffffff80c02d49:       4c 89 23                mov    QWORD PTR [rbx],r12
ffffffff80c02d4c:       48 8b 5d e8             mov    rbx,QWORD PTR [rbp-0x18]
ffffffff80c02d50:       65 48 2b 1c 25 28 00    sub    rbx,QWORD PTR gs:0x28
ffffffff80c02d57:       00 00
ffffffff80c02d59:       75 09                   jne    ffffffff80c02d64 <rsa_set_pub_key+0x8e>
ffffffff80c02d5b:       48 83 c4 50             add    rsp,0x50
ffffffff80c02d5f:       5b                      pop    rbx
ffffffff80c02d60:       41 5c                   pop    r12
ffffffff80c02d62:       5d                      pop    rbp
ffffffff80c02d63:       c3                      ret
ffffffff80c02d64:       e8 19 ce 6e ff          call   ffffffff802efb82 <_einittext+0x1efb82>

Disassembly of section .init.text:

Disassembly of section .altinstr_aux:

Disassembly of section .altinstr_replacement:

Disassembly of section .exit.text:

Reviewing your blog post you did find rsa_parse_pub_key symbol. I would appreciate so much if you can help me about understanding how that symbol is not found as I believe is the same Kernel that you used for your decryption research.

Thanks!

[gui-randori7542]

Hello @PeterGabaldon,
First of all, sorry for the late reply and thank you for your contribution.

Concerning your issue, let's check we work on the same firmware version:

➜   user@debian ~/merge sha512sum FGT_VM64-v7.4.7.M-build2731-FORTINET.out.ovf.zip
031b27ecc65b53cc6f3c21d903e4f42e63848971d2f074b663ccbffcaeb15b010d372fed43cfeacc1415af96396e90bd9df4892a40079f333b68bd0ebcb85c8f  /media/user/Extreme SSD/Fortigate/FGT_VM64-v7.4.7.M-build2731-FORTINET.out.ovf.zip

My version seems to be exactly the same as yours: the SHA-512 hash matches the hash provided by Fortinet on download page (cf. your screenshot).

Then, I extracted the kernel flatkc by mounting the VMDK disk and launched vmlinux-to-elf against it:

(venv) ➜   user@debian ~/merge sha256sum flatkc
c3edc107fddfc254970d61233d6ff4bbf4b86d25324252b1b2835ff7154caaa2  flatkc
(venv) ➜   user@debian ~/merge python ~/tools/vmlinux-to-elf/vmlinux-to-elf flatkc flatkc.kernel
[+] Kernel successfully decompressed in-memory (the offsets that follow will be given relative to the decompressed binary)
[+] Version string: Linux version 4.19.13 (root@build) (gcc version 10.3.0 (GCC)) #1 SMP Mon Jan 20 18:15:22 America 2025
[+] Guessed architecture: x86_64 successfully in 3.91 seconds
[+] Found kallsyms_token_table at file offset 0x013d14b8
[+] Found kallsyms_token_index at file offset 0x013d17f8
[+] Found kallsyms_markers at file offset 0x013d0f50
[+] Found kallsyms_names at file offset 0x0134bf78
[+] Found kallsyms_num_syms at file offset 0x0134bf70
[i] Negative offsets overall: 99.9955 %
[i] Null addresses overall: 0.00452714 %
[+] Found kallsyms_offsets at file offset 0x01320d20
[+] Successfully wrote the new ELF kernel to flatkc.kernel
(venv) ➜   user@debian ~/merge sha256sum flatkc.kernel
d5b2214a0fa39d44b975903705f6499f0dd11cc993c4a7dc5f6e1bc8a0ee565e  flatkc.kernel
(venv) ➜   user@debian ~/merge objdump -M intel -d --disassemble=rsa_parse_pub_key flatkc.kernel

flatkc.kernel:     file format elf64-x86-64


Disassembly of section .text:

ffffffff8055d2ac <rsa_parse_pub_key>:
ffffffff8055d2ac:	55                   	push   rbp
ffffffff8055d2ad:	48 89 e5             	mov    rbp,rsp
ffffffff8055d2b0:	89 d1                	mov    ecx,edx
ffffffff8055d2b2:	48 89 f2             	mov    rdx,rsi
ffffffff8055d2b5:	48 89 fe             	mov    rsi,rdi
ffffffff8055d2b8:	48 c7 c7 70 e2 23 81 	mov    rdi,0xffffffff8123e270
ffffffff8055d2bf:	e8 9c be 0a 00       	call   ffffffff80609160 <asn1_ber_decoder>
ffffffff8055d2c4:	5d                   	pop    rbp
ffffffff8055d2c5:	c3                   	ret

Disassembly of section .init.text:

Disassembly of section .altinstr_aux:

Disassembly of section .altinstr_replacement:

Disassembly of section .exit.text:

The rsa_parse_pub_key symbolized function exists.
However, we can observe during vmlinux-to-elf execution that most of offsets match with your provided output except the following:

[+] Found kallsyms_offsets at file offset 0x01320d20

You don't have the same offset. Maybe in your case vmlinux-to-elf is not able to reconstruct kallsyms symbols table, but why?
Moreover, you have a high "Null addresses overall" rate (what does it mean?). Maybe first check the hash of your input file flatkc, to be sure we have the same?

The vmlinux-to-elf version I use refers to the commit fa5c9305ae1c4bbcd2debabb810e7613def690a7 (yes, a old version).
Maybe meanwhile a regression bug has been inserted? You should also give a try to this specific version.

If this finally works for you, can you confirm that the script also works for FortiGate v7.4.6?

I look forward to seeing your results.

Thank you

[PeterGabaldon]

Hello @gui-randori7542 ,

Do not worry about late replay, I am very grateful about your comments and for have invested time in my issue 😊.

We are working on the same firmware versions yeah.

PS C:\Users\Peter\Downloads> Get-FileHash -Algorithm SHA512 .\FGT_VM64-v7.4.7.M-build2731-FORTINET.out.ovf.zip | select Hash


Hash
----
031B27ECC65B53CC6F3C21D903E4F42E63848971D2F074B663CCBFFCAEB15B010D372FED43CFEACC1415AF96396E90BD9DF4892A40079F333B68BD0EBCB85C8F


PS C:\Users\Peter\Downloads>

Here is SHA512 of flatkc and rootfs. I belive we match, please confirm that. The flatkc.v7.4.7.elf is the out one from vmlinux-to-elf

$ sha512sum flatkc.*
6e691e214b75077c9662b27c438a8c0d171eeb5e7379d037deaae8a5ae08721b1aa7dc0dbae3e47fba875b576c2dd3e8441aa71eeb500f3212dfd3eff1d04299  flatkc.7.4.7
a8bc4760ee813039228a7f8146e08799e6310ce16b1c21222e6285b20f909abcd573d111ab8f575a2ac38b9e595918a7c2e4b2bd3cdd0754c1563bd06207071b  flatkc.v7.4.7.elf

In my case I was using latest commit. Here is the output when using the same commit as you.

$ git checkout fa5c9305ae1c4bbcd2debabb810e7613def690a7 
Note: switching to 'fa5c9305ae1c4bbcd2debabb810e7613def690a7'.

You are in 'detached HEAD' state. [...]

HEAD is now at fa5c930 Update the supported Linux kernel version in the README file
peter@fuzzing:~/FGATE_Reversing/vmlinux-to-elf$ ./vmlinux-to-elf  ../flatkc.7.4.7 ../flatkc.v7.4.7.elf.old.commit
/home/peter/FGATE_Reversing/vmlinux-to-elf/vmlinux_to_elf/vmlinuz_decompressor.py:280: SyntaxWarning: invalid escape sequence '\d'
  if not search(b'Linux version (\d+\.[\d.]*\d)[ -~]+', input_file):  # No kernel version string found
/home/peter/FGATE_Reversing/vmlinux-to-elf/vmlinux_to_elf/elf_symbolizer.py:98: SyntaxWarning: invalid escape sequence '\('
  """
/home/peter/FGATE_Reversing/vmlinux-to-elf/vmlinux_to_elf/kallsyms_finder.py:221: SyntaxWarning: invalid escape sequence '\d'
  regex_match = search(b'Linux version (\d+\.[\d.]*\d)[ -~]+', self.kernel_img)
[+] Kernel successfully decompressed in-memory (the offsets that follow will be given relative to the decompressed binary)
[+] Version string: Linux version 4.19.13 (root@build) (gcc version 10.3.0 (GCC)) #1 SMP Mon Jan 20 18:15:22 America 2025
[+] Guessed architecture: x86_64 successfully in 2.59 seconds
[+] Found kallsyms_token_table at file offset 0x013d14b8
[+] Found kallsyms_token_index at file offset 0x013d17f8
[+] Found kallsyms_markers at file offset 0x013d0f50
[+] Found kallsyms_names at file offset 0x0134bf78
[+] Found kallsyms_num_syms at file offset 0x0134bf70
[i] Negative offsets overall: 99.9955 %
[i] Null addresses overall: 0.00452714 %
[+] Found kallsyms_offsets at file offset 0x01320d20
[+] Successfully wrote the new ELF kernel to ../flatkc.v7.4.7.elf.old.commit

Now yes, kallsyms appears to be well constructed.

$ objdump -M intel -d --disassemble=rsa_parse_pub_key flatkc.v7.4.7.elf.old.commit 

flatkc.v7.4.7.elf.old.commit:     file format elf64-x86-64


Disassembly of section .text:

ffffffff8055d2ac <rsa_parse_pub_key>:
ffffffff8055d2ac:       55                      push   rbp
ffffffff8055d2ad:       48 89 e5                mov    rbp,rsp
ffffffff8055d2b0:       89 d1                   mov    ecx,edx
ffffffff8055d2b2:       48 89 f2                mov    rdx,rsi
ffffffff8055d2b5:       48 89 fe                mov    rsi,rdi
ffffffff8055d2b8:       48 c7 c7 70 e2 23 81    mov    rdi,0xffffffff8123e270
ffffffff8055d2bf:       e8 9c be 0a 00          call   ffffffff80609160 <asn1_ber_decoder>
ffffffff8055d2c4:       5d                      pop    rbp
ffffffff8055d2c5:       c3                      ret

Disassembly of section .init.text:

Disassembly of section .altinstr_aux:

Disassembly of section .altinstr_replacement:

Disassembly of section .exit.text:
peter@fuzzing:~/FGATE_Reversing$

If this finally works for you, can you confirm that the script also works for FortiGate v7.4.6?

Yes, I have tried the script with my modifications and can confirm that it also works for 7.4.6.

So, it appears that the problem is vmlinux-to-elf.

Again, thanks for your time.