adam/meshing-around
1
0 Fork 0
mirror of https://github.com/SpudGunMan/meshing-around.git synced 2026-10-30 22:01:57 +01:00
Code Issues 12 Packages Projects Releases Wiki Activity

Ollama AI Gemma 2B Accuracy Decreased #23

New Issue
Closed
opened 2026-10-05 01:16:28 +02:00 by adam · 3 comments
adam commented 2026-10-05 01:16:28 +02:00
Owner
Copy link
Copy source

Hey Team,

Just an FYI that the latest Git Pull has dramatically decreased the accuracy of data coming from Gemma. Where the LLM was previously unable to provide data the Google search augmented search is pulling wrong information. An example is there are apparently 9 seasons of The Rookie a tv show which is wrong. Also when asked how many medals New Zealand win at the 2024 Paris Olympics, she said 1.

I'm unclear at this stage if the removal of common words, which is new code is contributing to this or if it's the Google search results.

Great to see so much development going on with this project!

Hey Team, Just an FYI that the latest Git Pull has dramatically decreased the accuracy of data coming from Gemma. Where the LLM was previously unable to provide data the Google search augmented search is pulling wrong information. An example is there are apparently 9 seasons of The Rookie a tv show which is wrong. Also when asked how many medals New Zealand win at the 2024 Paris Olympics, she said 1. I'm unclear at this stage if the removal of common words, which is new code is contributing to this or if it's the Google search results. Great to see so much development going on with this project! <!-- github-sync-id:2507423103 -->
adam closed this issue 2026-10-05 01:16:28 +02:00
adam commented 2026-10-05 01:16:28 +02:00
Author
Owner
Copy link
Copy source

without google results the LLM couldn't accurately say what 2024 they dont have up to date info, the LLM are not perfect at all. BUT if you liked the old version you can still go back easily try changing the line in llm.py https://github.com/SpudGunMan/meshing-around/blob/7afd6bbbe91e17f718ca51a8e3b7e1cdb7fcd19a/modules/llm.py#L14

change this to False

without google results the LLM couldn't accurately say what 2024 they dont have up to date info, the LLM are not perfect at all. BUT if you liked the old version you can still go back easily try changing the line in llm.py https://github.com/SpudGunMan/meshing-around/blob/7afd6bbbe91e17f718ca51a8e3b7e1cdb7fcd19a/modules/llm.py#L14 change this to False <!-- github-comment-id:2332115367 -->
adam commented 2026-10-05 01:16:29 +02:00
Author
Owner
Copy link
Copy source

Thanks, I've implemented this and it seems better! - Another question if I may please. The LLM even with Google disabled doesn't seem to work without internet connectivity. Is this intentional? - I'd like to operate this in a Meshtastic Only environment where the R-Pi is offline.

Thanks, I've implemented this and it seems better! - Another question if I may please. The LLM even with Google disabled doesn't seem to work without internet connectivity. Is this intentional? - I'd like to operate this in a Meshtastic Only environment where the R-Pi is offline. <!-- github-comment-id:2333139474 -->
adam commented 2026-10-05 01:16:30 +02:00
Author
Owner
Copy link
Copy source

cleaned this up a lot, it was messy! Yes it should work fine offline.

fixed in https://github.com/SpudGunMan/meshing-around/commit/819c37bdec401eeb7363a8ba84da96af02c4ea04

cleaned this up a lot, it was messy! Yes it should work fine offline. fixed in https://github.com/SpudGunMan/meshing-around/commit/819c37bdec401eeb7363a8ba84da96af02c4ea04 <!-- github-comment-id:2333311788 -->
Sign in to join this conversation.
No Branch/Tag Specified
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
adam (Adam Wojcieszonek)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: adam/meshing-around#23