LUCIE: the 100% Open Source LLM

They're talking about us... ID L'info Durable

LUCIE, the 100% open source LLM developed with the OpenLLM France community and based on transparent data, has sparked a lively debate in recent days. Between criticism and enthusiasm, one thing is certain: French AI is being built, with its own challenges and ambitions!

  But what about training generative AI?

1.   The model must first learn the languages in which he or she is expected to converse.
    
2. AI will train itself to make links between ‘tokens’. A token represents a unit of data understood by an AI. It can be part of a sentence, a word or part of a word. 
    
3. The AI is then analysed by a human, who uses a question-and-answer game to correct its answers. This is the RHLF stage, or Reinforcement Learning from Human Feedback. It enables the AI to make links between the question asked and the knowledge it possesses, with the aim of providing an accurate answer.

LUCIE has not yet undergone this final stage, which is why it has been put online.

• Generative AI is developed in 3 phases:
  Pre-training: For LUCIE this is learning, it is fed with a huge amount of data, learning language structures and possible responses.
   
• Alignment: LUCIE teaches how to adopt the right behaviours and manage specific use cases.
   
•  Final fine-tuning: LUCIE is fine-tuned by analysing its responses, identifying biases or errors, to deliver the best performance before it goes live.

LUCIE has only just begun the alignment phase. Building sovereign AI takes time. But LUCIE's ambition is clear: to propose a French artificial intelligence model that is ethical and adapted to the needs of our education system.