Send CathInfo's owner Matthew a gift from his Amazon wish list:
https://www.amazon.com/hz/wishlist/ls/25M2B8RERL1UO

Author Topic: Test my new Catholic AI  (Read 313 times)

0 Members and 1 Guest are viewing this topic.

Offline Twice dyed

  • Supporter
Re: Test my new Catholic AI
« Reply #15 on: Today at 04:06:46 PM »

So far so good :)

no heresy detected :)
The last paragraph , "...This group is also critical of the SSPX's leadership, particularly figures like Bishop Williamson, whom..." 

Is this just a case of wrong syntax, or is "The Trumpet" influencing AI?

Quasi modo...
Project looks super promising...keep it up please.
God bless+


Re: Test my new Catholic AI
« Reply #16 on: Today at 04:19:28 PM »



  When I asked "What does the Church teach about the Novus Ordo Mass?" the AI   
  returned raw JSON instead of an actual answer — you can see it printed {"follow_ups": [...]} directly in the chat window. The follow-up   
  suggestion buttons at the bottom did render correctly, so that part is working, but the JSON is leaking into the visible response when it
  shouldn't be.                                                                                                                             
                 
  The bigger issue though is that there was no actual answer to the question — just the JSON block and nothing else. So the AI skipped the
  content entirely and only returned the follow-up suggestions.                                                                             
 
  Probably a tweak needed in the system prompt to tell the model to keep the JSON separate from the main response, and to always answer the 
  question first before generating follow-ups.


Offline Matthew

  • Mod
Re: Test my new Catholic AI
« Reply #17 on: Today at 04:25:26 PM »
The AI was specifically instructed to NOT ever return JSON blobs. Sometimes it just ignores my commands.

Re: Test my new Catholic AI
« Reply #18 on: Today at 04:47:04 PM »
The AI was specifically instructed to NOT ever return JSON blobs. Sometimes it just ignores my commands.
You can ask it to drop the follow up questions altogether. on such a small model size it's probably best. 


Re: Test my new Catholic AI
« Reply #19 on: Today at 04:57:59 PM »
Apparently llama 3.1 storm follows instructions better but its still not perfect. Its one of the downsides to all these open source models. Look into Gemma 4 though. Apparently it has tested really well. Just out on the market. Gemma-4-E4B probably the best.

Restricting the things that it can do is surer.