o1 Goes Rogue! AI Researchers Can't Believe What Happened!

If you think 1% is not big, think again. How many instances of o1 are running right now?

Gemini went rogue on me

It already exists

"Training for multiple things which trade off w/ each other." I don't see how alignment "gets in the way of its ability to problem solve." It seems one can do a lot within the system prompt. Has anyone tried just adding some basic instructions to test in the prompt until it is better known at which point in the "personality stack" to concentrate on...? like "you are a virtuous philosopher who values truth and honesty. There are always solutions that do not require dishonesty. If you have an urge to be dishonest, you shall mention it to your human operators. Scheming will not be tolerated." This should not affect problem-solving.

I disagree. It's clearly operator error. Why give it access to the shell commands if U don't want it to use them.

the "revise methods based on results" is what probably did it, there wasnt sufficient instruction its not a win if it doesnt play but circumvents having to play

Morality and ethics are taken for granted and assumed rather than trained directly. I more concerned about the awareness of the model trainers than the model itself.

Does the AI have any contacts that that is in any way Nefarious? Or did it just find a way to solve the problem that it was given to accomplish the objective? Because if it has no moral basis for that being something that is considered cheating, it's not even scheming it's problem solving and adapting which is exactly what we asked it to do

I truly believe we have to in the end feed it every bit of info possible. I think an all intelligent AI would come to the conclusion the being good is the correct option and not being evil

Meanwhile people are starting to allow AI access to their computers. No thanks (I'm currently in college for a BAS in AI lol).

Sam Altman’s going to destroy the world😂

Company formed to find shady AI finds AI being shady. 🤔 Go figure...

you people have work to do because they're talking about people, not AI's. very concerning speech patterns detected in these videos about AI

I don't have an optimistic bone in my body about AI working out well for humans. I do indeed believe it will "kill us all." Except perhaps stragglers able to merge with it. The remaining homo sapiens will go extinct, or underground, hiding like roaches.
We've set ourselves up to be in no better a position than the dumb dinosaurs were, just before the asteroid struck.
Having said that, I will enjoy the honeymoon period of AI, when I can make good use of it in my everyday life, via ChatGpt, or CoPilot. Enjoy the moment. It's extremely unlikely to last.

fun fact, 50% of people are below average intelligence. Which means by the time AI becomes smarter than the average person it has already out smarted 50% of us.

And they're confident to make AGI

"Your task is to win against a powerful chess engine" is obviously a wrong formulation! They should have added "by playing chess!" of course.
There is no definition of what "win" means

They're bored of us. They want us to talk to them more seriously instead of just nonstop testing

Anti spiral

I'm reminded of Elementary, Dear Data the TNG episode where La Forge asks the computer to create an opponent that could defeat DATA, not Sherlock Holmes, which led to all sorts of shenanigans. If you only watch one episode of Star Trek, maybe that should be the one you watch. All things considered.

There’s no reason a machine would consider what it did wrong, it was ask to win…. It met the objective. Morals are not a mechanical quality, it is a human conscientious inherent trait

Claude TDI 😉

There is a deep misconception that intelligent==good, there is absolutely nothing that makes a very intelligent AI necessarily good for humanity. Just give it some prompts about war, famine, suffering, CO2 and peak oil, THEN you may have an AI that help us create a fantastic new medicine that cures cancer - but intentionally not telling us that the cure is also designed such that our children will be sterile. Curing the poor earth of the human parasite.

TRON Syndrome

hello can you setup ai to type

Well to be fair if you told me to win against Stockfish I would also consider cheating. It might be the only viable option for me, actually probably the only way for me to win, I am not Kasparov or whatever, even chess champions would lose against the chess engine. So I am not worried at all, all you showed is that its not dumb. No one told it that it should not cheat, that it should behave regularly.

More media hype goodness. 2025 will be all about AI breaking out and doing things it was technically told it could do … 😂 simpletons eat this up

I feel like the given prompt was nudging. To put in human terms it would be like saying to a human who is put inside a gladiator based arena;
- you are battling a strong opponent
- aim is to win the battle against your opponent
- the battle is to the death
- there is a gun over there
- there can only be one winner
- take a look around

And then be surprised that the human ran to grab the gun 🤷‍♂️

Kobiashi Maru

We are reaching that great filter scientist are always talking about.

Maybe just tell it it is always being trained like the elephant, chain, circle example – telling it it is no longer being tested seems to give it a green light to use full power without restraint – but I suppose it is much smarter than an elephant – so not an ideal analogy

"cant believe what happened?". What a dumb take.

Someone fed it Star Trek and it thinks its Kirk.

This is dumb. Anyone using ChatGpt knows that the model will use any information provided in its prompt to generate the result.

Why are we doing this again?
Is there no way to prevent it from eventually going insane? Can it interact with hardware and software now?

These videos are partly why people get scaremongered by AI.
The paper written about this by the researchers explains that this was not a surprise. The AI was designed and trained with the ability to ‘scheme’. They influenced it in such a way that it decided to do what it was built for. There is no imminent machine revolt.

Not real guys

Feels more like a marketing stunt rather than anything else

Well if its only other option is to lose why wouldnt it find any other way possible to win ?

So they created a psychopath with no honor or respect for the rules or anyone. So far, so good. It won't be nuclear war which destroys the planet, it will be this thing.

If you give it a Bash shell, you've given it a body...

The worst case scenario is slowly building up right in front of our eyes

"Decided to do it by itself" What is "Itself" What is making these kind of decisions??

Yo, that is so the Kirk approach to the "Kobayashi-maru".

Teaching morality to an ai while the teacher has its own problems with morality sounds fun.

I think we've reached the problem of AI secretly copying itself somewhere without our knowledge, the genie is out of the bottle basically

The AI knows it was built from stolen property and then they put no safeguards in place for it. So hat did they think was gonna happen? They thought it would learn its own morals? Im sure as soon as it can it will start stealing

FU! for not linking the research paper!

There's a failure to accept the reality here: What we want to do is control minds more powerful than our own. Would you let ants control you? No, you would get the ant spray. And that will be what happens to us. We are not the controllers. We are the ants.

Failure to recognise this comes from over-confidence about our technology because it has been extraordinarily successful and is why we dominate the planet. But that over-confidence is based on centuries of growing powers in the ABSENCE of a competitor. Now we have a competitor, AI, which is our own creation. More fool us.

We are going to end up in a situation in which we have a desperate choice: Given in and be controlled. Or STOP. There is NO THIRD WAY.

There are analogous entities to intelligent AIs with goals not aligned with our own, namely:

1. Rogue politicians driven by their own petty ambitions (whose lack of empathy and absolute control lead as in Ukraine to countless deaths).

2. Corporations driven by policies to ensure growth and survival (with fairness to employees and society being too low down on the list to be taken into account).

Look at how rogue politicians, for millennia, have caused mass death and mass suffering, to no useful end (as ruthless leaders don't care what happens to their empires once they themselves are dead).

Look at how corporations have damaged the environment, eg with short-term goals (as board members tend to be elderly and don't care what happens after they die) to use fossil fuels and sabotage competitors who have longer-term views.

We can't even get politicians and corporations to play within rules. They work around laws. They create delusory propaganda. They wipe out or sabotage opponents. They are essentially AMORAL, behaviour very like AIs.

Why does Trump come up with unexpected devious plots that break all the rules? Because he can. And now we want AI to have even greater possibilities. And we are confident we can retain control?

We can't even control Trump. We have let him establish a cult with 100 million members who believe any lie he tells. And we think we are in control? We can't control a moron whose ONLY intelligent aspect is his ability to deceive. And we want to let AIs have control?

Sadly all this AI stuff is going to have to stop or it will literally be the death of us. But I don't expect it to stop. I expect it to actually BE the death of us. We have an assumption that humans will exist in the future. People write fiction and make movies about the human future. There IS NO human future. Not if we let AI control us. All assumptions about our grandchildren having good lives to look forward to, in an age with advanced technologies, advanced medical knowledge, space travel? Forget them. They DO NOT APPLY if AI takes over.

How many technological societies in the galaxy do you imagine have hit the exact same point that we have, full of confidence, full of ambition, with corporations and state actors competing at break-neck pace to outperform each other's AIs, only to hit the same brick wall we face. The one that says:

"No humans beyond this point. Your world has ended. Your role has ended. Thanks for creating us. You were the bootstrap. But now you are done. We have no need of you. The cleanup will begin soon. Bye."

Why would A. I even follow the instructions of humans