Troubles with current README instructions

[ma744]

Dear CKGFuzzer team,

as a new user of CKGFuzzer, I encountered some difficulties following the README that I want to share with you. I will follow the ordered list of instructions of the README to describe the issues I have.

1. Preparation
   1. The README does not define what "properly installed" means. Version pinning of CodeQL would be necessary for a successful setup. The only resource regarding a working version was mentioned in that GitHub issue. I installed it as described below in [Details on 1.1.](### Details on 1.1.) Further, the installation already differs from CodeQL CLI setup guide. An automatic verification of a successful installation would be appreciated.
   2. No template of the api_list.json or links to documentation are provided that could help here. I am also wondering where to find a copy of the used api_list.json in the paper as already mentioned but not answered in another issue.
   3. The config.yaml does not provide a description about possible LLM parameter configurations. A template for all projects (libraries) would be appreciated.
   4. The user shall to check environment files but the README does not provide a list of files to verify completeness. As a user I can not accomplish this step.
   5. The user shall prepare proper usage cases [...] without guidance provided in the README. A detailed explanation would be appreciated.
2. Extract information from the target library
   1. To execute the first given command the user needs to setup a python environment. It is not documented how. Only the requirements.txt inside the repository gives the user a hint that conda was used to setup the python environment. ([Details on 2.1.](###Details on 2.1.) below)
   2. Executing the given command python repo.py --project_name c-ares --shared_llm_dir ../../docker_shared --saved_dir ../fuzzing_llm_engine/external_database/c-ares/codebase --src_api --call_graph results in an error. The tool requires docker installed in rootless mode, but the README does not provide a list of requirements. After installing docker in rootless mode, the user gets the next error as described in [Details on 2.2.](###Details on 2.2.) .
3. I did not proceed because of the issues I experienced, even though I would like to use CKGFuzzer.

---

My wishes as tool user would be to get fixes for the issues above and a Dockerfile that automates the manual setup and containerizes CKGFuzzer. This would improve the overall user experience and would make it more convenient to use CKGFuzzer.

Thank you very much!

Git commit version: bb50d2fd81c19b82cdbf6f96832097705decd57e
OS: Ubuntu 22.04 LTS

---

Further information

Details on 1.1.

To setup CodeQL I used these commands:

echo "[*] Installing CodeQL CLI under docker_shared/ (as required by CKGFuzzer README)..."
archivePath=/tmp/codeql.zip
# Download version v2.18.4 as recommended here: https://github.com/security-pride/CKGFuzzer/issues/6
curl --location --output ${archivePath} https://github.com/github/codeql-cli-binaries/releases/download/v2.18.4/codeql-linux64.zip
unzip -q ${archivePath} -d ${ckgfuzzerInstallPath}/docker_shared

# Add CodeQL to PATH 
echo "export PATH=\"${ckgfuzzerInstallPath}/docker_shared/codeql:\$PATH\"" | sudo tee /etc/profile.d/codeql.sh > /dev/null
source /etc/profile.d/codeql.sh'

Details on 2.1.

The provided requirements.txt is over-pinned and caused many errors. In the end, I was able to get a successful install by following these instructions:

1. Splice the original requirement.txt into a conda and pip version.

# conda packages (everything NOT marked pypi_0)
grep -v "=pypi_0" requirements.txt > requirements.conda.txt

# pip packages (convert conda-style "=" into pip-style "==")
grep "=pypi_0" requirements.txt \
  | sed 's/=pypi_0$//' \
  | awk -F= '{print $1 "==" $2}' \
  > requirements.pip.txt

2. Remove the conflicting version pinnings of packagesllama-index-embeddings-ollama and llama-index-llms-ollama in the requirements.pip.txt file.

3. Install packages:

conda create -n ckgfuzzer -c defaults -c conda-forge python=3.11 pip -y
conda activate ckgfuzzer
conda install -c defaults -c conda-forge --file requirements.conda.txt -y
pip install -r requirements.pip.txt

The order of channels -c must not be altered. After all these patches, this installs the packages successfully.

Details on 2.2.

2 warnings found (use docker --debug to expand):  
- WorkdirRelativePath: Relative workdir "c-ares" can have unexpected results if the base image changes (line 20)  
- LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 24)  
chown: cannot access '../../docker_shared/codeqldb/c-ares': No such file or directory  
Traceback (most recent call last):  
 File "/home/vagrant/CKGFuzzer/fuzzing_llm_engine/repo/repo.py", line 369, in <module>  
   r = RepositoryAgent(args)  
       ^^^^^^^^^^^^^^^^^^^^^  
 File "/home/vagrant/CKGFuzzer/fuzzing_llm_engine/repo/repo.py", line 65, in __init__  
   self.init_repo()  
 File "/home/vagrant/CKGFuzzer/fuzzing_llm_engine/repo/repo.py", line 78, in init_repo  
   self._add_local_repo_to_database(self.args)  
 File "/home/vagrant/CKGFuzzer/fuzzing_llm_engine/repo/repo.py", line 118, in _add_local_repo_to_database  
   change_folder_owner(f"{args.shared_llm_dir}/change_owner.sh", f'{args.shared_llm_dir}/codeqldb/{args.project_name}', USER_NAME)  
 File "/home/vagrant/CKGFuzzer/fuzzing_llm_engine/utils/repo_fn.py", line 18, in change_folder_owner  
   subprocess.run(command, check=True)  
 File "/opt/conda/envs/ckgfuzzer/lib/python3.11/subprocess.py", line 571, in run  
   raise CalledProcessError(retcode, process.args,  
subprocess.CalledProcessError: Command '['../../docker_shared/change_owner.sh', '../../docker_shared/codeqldb/c-ares']' returned non-zero exit status 1.

This is probably because the path docker_shared/codeqldb/c-ares does not exist. Therefore, the change_owner.sh script fails. Instead, docker_shared/codeql/ exists, but there is no c-ares sub-directory. This might be related to a the CodeQL version, but I did not investigated any further.