BEGIN:VCALENDAR PRODID:-//eluceo/ical//2.0/EN VERSION:2.0 CALSCALE:GREGORIAN BEGIN:VEVENT UID:3572370ecc241f531050042ada48e4d1 DTSTAMP:20260520T160915Z SUMMARY:Master thesis presentation: Benchmarking Large Language Models for Vulnerability Detection: Comparing Local and Cloud LLMs DESCRIPTION:Contact: susanna.lonnqvist@eit.lth.se\n\nBenchmarking Large Lan guage Models for Vulnerability Detection: Comparing Local and Cloud LLMsAb stract: This thesis investigates the possibility of utilizing locally fine -tuned LLMs in order to discover and flag memory related security flaws in C and C++- code. Five locally fine-tuned models have been examined and co mpared to each other\, their non-fine-tuned versions\, as well as propriet ary cloud models. The models were fed functions taken from C/C++-projects\ , and were asked to determine whether the function in question was vulnera ble.Two different prompting methods were used during the evaluation\, whic h were zero-shot prompting and few-shot prompting. After each evaluation\, performance metrics such as accuracy and F1-score were calculated. We sho w that while fine-tuning enhanced the performances of the local models wit h respect to F1-score\, their ability to detect vulnerabilities remained u nsatisfactory. The highest performing model\, CodeLlama 7B\, achieved a F1 -score of only 0.12. However\, as the cloud models\, which are orders of m agnitude larger in parameter size and with more extensive pre-training\, d id not outperform this\, it indicates that the methods utilized in the the sis were suboptimal. \;Supervisor: Christian GehrmannExaminer: Thomas Johansson\n\nMore information about the event: https://www.eit.lth.se/en/c alendar/master-thesis-presentation-benchmarking-large-language-models-vuln erability-detection-comparing DTSTART;TZID=GMT:20260526T081500 DTEND;TZID=GMT:20260526T090000 LOCATION:E:3139 END:VEVENT END:VCALENDAR