Compiler setup

Windows

Requirements:

VisualStudio 2019 and higher

macOS

Requirements:

Xcode 14.2 and higher

Linux

Requirements:

clang

Building the sample

Move to the cpp folder, then run the command corresponding to your OS as described below.

Windows

cl ailia_llm_sample.cpp ailia_llm.lib

macOS

clang++ -o ailia_llm_sample ailia_llm_sample.cpp libailia_llm.dylib -Wl,-rpath,./ -std=c++17

Linux

export LD_LIBRARY_PATH=./

g++ -o ailia_llm_sample ailia_llm_sample.cpp libailia_llm.so

Model file download

Download the specified model file and place it in the models folder.

https://huggingface.co/bartowski/gemma-2-2b-it-GGUF/resolve/main/gemma-2-2b-it-Q4_K_M.gguf?download=true

Running the sample

Run the sample by executing the command below.

./ailia_llm_sample

Platform-specific remarks

Release the 'downloaded' attribute in MacOS

Binaries downloaded from the browser in macOS may have a 'downloaded' attribute and may not be able to be executed. In that case, it is possible to remove the attribute by right-clicking on the .dylib file and executing it. You can also release the 'downloaded' attribute by executing the following command from the command line:

xattr -d com.apple.quarantine libailia_llm.dylib