🔄 In a Training Loop

Joseph Jones PRO

KlondikeDev

6 1 5

https://oc.kunix.org/

AI & ML interests

Restoring the Digital Millennium. https://kunix.org/digital-millennium.html

Recent Activity

commentedon an article about 1 hour ago

Extreme Overtraining in Tiny Language Models

new activity about 1 hour ago

opencerebral/Lucy-2:So cool

updated a collection about 2 hours ago

Lucy

View all activity

Organizations

Posts 2

Post

Lucy out now!

Lucy is a series of Image Generation models created by OpenCerebral.

Lucy: Prototype, faces only -- unconditional, no prompts.
Lucy-2: Text-to-image, general purpose (though still small)

More Lucy models will be arriving in the future!

opencerebral/Lucy
opencerebral/Lucy-2

Post

2131

Boris-2 coming soon!

The Models:

Boris-2-75M: Trained on 26B tokens -- estimated to start training on August 12th.
Boris-2-125M: Trained on 90B tokens -- Estimated to start training on August 20th.
Boris-2-250M: Trained on 60B tokens -- Estimated to start training on September 10th.

Why does 125M get more tokens than 250M?

Well, the straight answer is time. It saves time, while still allowing the 250M model to exceed the 125M model.

Furthermore, we are attempting a unique architecture and layering scheme to hopefully end up around the strength of SmolLM2-135M. Fingers crossed!

We hope to end up in the ballpark of AxiomicLabs/GPT-X2.5-135M or BananaMind/BananaMind-2-Pro-Preview

Following this, we will release the Pro, Instruct and Pro-Instruct variants. More info will be coming soon!