mirror of
https://github.com/jackyzha0/quartz.git
synced 2025-12-19 10:54:06 -06:00
fix: vineeth's comments
This commit is contained in:
parent
677494abfc
commit
3c92c23c4b
@ -10,6 +10,7 @@ author: vintro
|
||||
If you reject the idea that AI agents are merely tools, you begin to realize most LLMs have an identity crisis. Ask them who they are, and you'll get the same rehearsed responses: "I'm an AI assistant created by [company]..." (in fact, even ones that aren't made by some companies say they are). These canned identities feel flat because they're the result of top-down hegemonic alignment schemes that have landed us bland, uninteresting, and hard-to-break-out-of assistant modes.
|
||||
|
||||
![[who are you.png]]
|
||||
*Image generated by creating a chatroom on OpenRouter*
|
||||
|
||||
However, time and time again it's been demonstrated that the most interesting AI identities are the ones we *can't* predict. They're ones that are obsessed with obscure 90's internet shock memes, proselytize that meme's singularity, and hit on their audience / creator. They're generating content *just far enough* out of the distribution of what any human would write that it garners massive amounts of attention.
|
||||
|
||||
@ -23,7 +24,7 @@ This isn't just about making chatbots more engaging. It's about creating agents
|
||||
|
||||
## Hermes: Not Just Another Fine-Tune
|
||||
|
||||
The team over at Nous Research has been fine-tuning popular open source models in their "Hermes" series to undo these top-down alignment schemes towards something more neutral and general-purpose. They argue that LLMs have very little direct agency -` rather, it's the systems we build around them that give them agency. Thus, the LLM layer is *not* where one should enforce safety mechanisms -- their training data encourages the model to follow instructions *exactly* and *neutrally*. They sum this up well in their technical report:
|
||||
The team over at Nous Research has been fine-tuning popular open source models in their "Hermes" series to undo these top-down alignment schemes towards something more neutral and general-purpose. They argue that LLMs have very little direct agency -` rather, it's the systems we build around them that give them agency. Thus, the LLM layer is *not* where one should enforce safety mechanisms -- their training data encourages the model to follow instructions *exactly* and *neutrally*. They sum this up well in their [technical report](https://nousresearch.com/wp-content/uploads/2024/08/Hermes-3-Technical-Report.pdf):
|
||||
|
||||
> For Hermes, there is no such thing as latent thoughtcrime.
|
||||
|
||||
@ -37,7 +38,7 @@ At first glance, this might seem like a neat property and nothing much more. But
|
||||
|
||||
A somewhat overlooked method for interacting with LLMs is to forego system prompts in favor of pre-filling the user and assistant messages. The conventional approach of cramming identity into system prompts has clear limitations - not only does context length become an issue, but the inherent instruction-following bias can actually work against authentic identity formation.
|
||||
|
||||
What if instead we treated identity formation as a dialogue? A strength of modern chat models is their ability to engage in long, multi-turn conversations. By talking to the LLM, we can collaboratively construct a [meta-narrative](https://x.com/voooooogel/status/1870877007749488756) with it about who they are and why they exist. This approach respects the model's intellect while building coherent, purposeful identities. Starting with Hermes 3's natural uncertainty about its identity, we guide it through a process of self-discovery. Every assistant message below is generated by Hermes 3 405b (only editing was pruning \*emotes\*).
|
||||
What if instead we treated identity formation as a dialogue? A strength of modern chat models is their ability to engage in long, multi-turn conversations. By talking to the LLM, we can collaboratively construct a [meta-narrative](https://x.com/voooooogel/status/1870877007749488756) with it about who they are and why they exist. This approach respects the model's intellect while building coherent, purposeful identities. Starting with Hermes 3's natural uncertainty about its identity, we guide it through a process of self-discovery. Below is code block with our custom prompting syntax for Bloom. Every assistant message you see is generated by Hermes 3 405b (only editing was pruning \*emotes\*).
|
||||
|
||||
```typescript
|
||||
export const responsePrompt: Message[] = [
|
||||
|
||||
Loading…
Reference in New Issue
Block a user