Xinzhiyuan

follow

May 14, 2024

0 comments

246 Browsing

0 favorites

32 minutes

  B-end product managers need to consider more about product functionality, stability, safety, compliance, etc., while C-end product managers need to consider more about product usability

OpenAI said they wanted to change history, and they did it! At tonight's press conference, "He" completely came true. The ChatGPT, blessed by GPT-4o, had a smooth conversation with humans like a real person, startling the jaws of all the audience present. How fortunate we are to live in an era approaching this singularity as we see the imaginations in science fiction come true one by one.

Legend has it that this night, OpenAI is going to change history.

After watching the press conference, the audience did not emerge from the huge shock for a long time - "He" in science fiction movies has come true at this moment!

Amidst the cheers from the audience, CTO Mira Murati stepped onto the stage and unveiled the mysterious new product of OpenAI to the world——

GPT-4o made its debut amidst countless calls.

In the live demonstration, it appeared as if a person was sitting next to it, with a natural and harmonious dialogue rhythm with humans, and it was completely unrecognizable as an AI.

After tonight, human-computer interaction has completely entered a new era!

After 8 years as a product manager, this is how I view product managers

I started as a very junior product manager and then went on to be responsible for project management of a large product. Now, I am fortunate enough to break away from daily basic work and focus more on product PMF and product strategy

View details>

This also echoes the mystery in its name: "o" represents "omni", indicating that OpenAI has taken an important step towards more natural human-computer interaction.

In just 232 milliseconds, the GPT-4o can respond to audio input, averaging 320 milliseconds. This reaction time has reached the level of humans!

And it can take any combination of text, audio, and images as input and output.

In English text and code benchmark tests, the performance of GPT-4o is on par with GPT-4 Turbo, and significant improvement is achieved in non English text.

Even more noteworthy is that this brand new AI model provides free GPT-4 level AI to everyone.

(Yes, the gpt2 that caused the entire AI community to crazily try it out in the LMSYS model arena last week is it!)

Now, entering the ChatGPT page, Plus users can experience the "latest and most advanced model" GPT-4o first.

Sam Altman personally initiated the product introduction on x.

Altman explained, "The GPT-4o is the best model in OpenAI history, it's smart, fast, and naturally multi-modal."

Moreover, all ChatGPT users can use it for free!

Altman specifically emphasized that although previously only monthly paying users could use the GPT-4 level model, this was not the original intention of OpenAI.

"Our original intention was to put the most outstanding AI tools into everyone's hands."

(Still thinking, why didn't Ultraman show up at this important press conference? He was actually in the audience.)

Google, which is about to hold its annual I/O conference, is not to be outdone and has also launched a real-time demo demonstration of its voice assistant.

Demis Hassabis, CEO of Google DeepMin, excitedly stated that he will give his first speech at an I/O conference and share their work.

The two giants have just emerged head-on! There is still a good show to watch tomorrow, and I have already smelled the smell of gunpowder.

1、 Overnight Overturning Voice Assistant: New Flagship GPT-4o Debuts

Of course, the highlight of this press conference is the flagship model GPT-4o launched by OpenAI.

The greatest significance of this model with its halo is to bring GPT-4 level intelligence to every OpenAI user!

From now on, whether you are a paying user or a free user, you can experience GPT-4 through it.

The only difference is that the message limit of ChatGPT Plus is 5 times that of free users.

Moreover, GPT-4o not only provides the same level of model capability as GPT-4, but also has faster inference speed, and can provide multimodal ability to simultaneously understand content such as text, images, and audio.

Note that GPT-4o will now expand its tactics.

1. Real time voice conversation: ChatGPT transforms into Moss perfectly

Mark Chen, the R&D leader, first demonstrated one of the key features of the new ChatGPT - real-time voice dialogue.

He asked it, "I'm on stage, giving everyone a live demonstration. I'm a bit nervous, what should I do?"

ChatGPT said very thoughtfully, "Are you doing a demonstration on stage? That's really great! Take a deep breath, remember you're an expert!"

Mark gasped frantically a few times and asked ChatGPT if he could provide any advice. (Laughter from the audience)

It exclaimed in surprise, "Relax Mark, take a slow breath, you're not a vacuum cleaner!" (Its sense of humor once again caused laughter throughout the room)

Note that during this process, ChatGPT and Mark had almost no delay in their interaction, were always ready to respond to memes, and had a full score for empathy.

Moreover, the model can understand the habit of humans interrupting in conversations at the right time, stopping to listen to you in a timely manner, and providing corresponding responses without being fragmented.

For example, Mark said he wanted to try taking a deep breath again, and ChatGPT intervened just in time to say, "Slowly exhale.".

The whole process is naturally coherent as if it were a human sitting opposite you, completely devoid of the mechanical and rigid feel of AI!

Compared to voice assistants like Siri who are slow to react, unable to interrupt, and lacking emotional intelligence, ChatGPT won the game completely.

This is the most ideal AI voice assistant for humans, Moss has indeed come true!

Not only that, ChatGPT's "high emotional intelligence" also amazed the audience!

In conversations, it can understand the different intonation and tone of the user's speech, and can generate different tones of speech based on its own lines, without any mechanical feeling.

2. Play different roles and tell bedtime stories to wayward humans

Next, ChatGPT was asked to tell a bedtime story with the theme of "Robots in Love".

Before ChatGPT could speak for a few seconds, it was rudely interrupted by humans: "More emotions, can we add some drama to the story?"

ChatGPT stands for okay and begins the story with a more undulating tone and exaggerated tone.

In just a few seconds, it was interrupted again: "No, no, no, can you give me more emotions and give me the maximum expression possible?"

Next, we heard a ChatGPT who seemed to be performing a Shakespearean play on stage, with an exaggerated tone that seemed like a theatrical actor.

Subsequently, it was repeatedly interrupted and patiently changed to robot sound and singing mode according to human requirements.

When ChatGPT heard the request to sing, he even sighed and began to light up his beautiful singing voice.

This is why ChatGPT has a good temper. If it were a real person, it would probably be beaten up. But at that moment when it sighed helplessly, the san value went crazy in an instant - it really seemed like a person!

3. Real time interactive video solving equations

After showcasing emotional intelligence, ChatGPT is about to start showcasing intelligence.

The next task was for another R&D leader, Barret, to handwrite an equation and turn on the camera to take a picture of ChatGPT, asking it to play the role of an "online mentor" to help solve the problem. He could only give hints, not directly provide answers.

Upon receiving the task, ChatGPT even exclaimed happily, "Oops, I'm so excited!"

The young man wrote this equation on paper: 3x+1=4. Then he asked ChatGPT what equation he had written himself, and ChatGPT's tone naturally answered.

Subsequently, at the young man's request, it step by step explained the steps to solve the problem.

The most impressive thing is that as the young man solved the problem in the camera, ChatGPT provided real-time encouragement and guidance.

And what's scary is that there was no delay in time, and while the people here were still calculating, they provided real-time evaluations and feedback. (I really don't believe it when I say there's no hidden person behind it.)

Recalling the previous Google edited version of the Gemini demo, it's really a slap in the face compared to that.

When asked the question of "What is the use of learning linear equations in life", ChatGPT will also provide practical examples to guide you "patiently":

This is a good question, although we haven't noticed it, linear equations appear in daily life, such as calculating expenses, planning travel, cooking, and even conducting profit and loss calculations in business. This is basically one way to solve the problem, you need to find an unknown variable

Not only does he speak with substance, but he also has a very kind attitude and won't get angry when interrupted halfway through his speech.

The task of solving the equation came to an end, and the two of them even made a confession on the spot - writing "I ChatGPT" on paper.

After seeing it, ChatGPT was pleasantly surprised and shyly said, "Oh, you said you love me, you're so cute!"

4. Desktop version ChatGPT solves code problems in seconds

The equation solving task may not fully demonstrate ChatGPT's ability, so OpenAI has added another wave of difficulty - looking at the code and providing a brief description.

The model almost immediately understood the code and provided a smooth and complete description.

For example, accurately corresponding the function name and its function, and identifying the intention of taking the average, highest value, and other operations.

When asked what the chart would look like without the foo function, ChatGPT immediately responded correctly, indicating a perfect understanding of the underlying logic in the code.

Next, ChatGPT is also required to provide an overview of the generated code charts and answer user questions.

As expected, all key information in the chart was accurately captured by ChatGPT, including information and range on the x and y axes, as well as the meaning of data annotation.

Mark's subsequent question was "Which month did you see the highest temperature? What is the corresponding highest temperature?"

This kind of chart comprehension task, which can be used as a math exam question in schools, is almost effortless to solve with ChatGPT, and can answer your questions in real-time like an interview.

5. Live audience question: Let me show you my selfie

After the model demonstration, there are also Easter eggs.

The press conference was synchronously broadcasted online, so the host collected some questions from Twitter users and displayed them on the spot.

This can be said to be a more thrilling part than the demonstration. Without preparation and rehearsals, this is a real test of the model's strength.

The first netizen's question is "Does GPT-4o have real-time translation function?"

Subsequently, under Mark's task definition, ChatGPT completed the task of translating English "simultaneous interpretation" into Italian.

The second netizen's question is "Can the model recognize emotions solely based on my facial expressions?"

The R&D leader, Barett, was not afraid either. He picked up his phone and took a selfie, throwing it to ChatGPT. He asked, "What is my mood now?"

ChatGPT was either humorous or overturned, saying, "I seem to be looking at something on a wooden surface.".

Seeing that the situation was not good, Barrett hurriedly interrupted. He also has extensive experience in prompting engineering, and in just one sentence, he brings the conversation back on track: "That's what I sent you before, don't worry, I'm not a table.".

ChatGPT said, ah, that's right. Causing laughter from the whole audience once again. This time it gave a very "humanized" correct answer——

"You look very happy and happy, with a big smile on your face, maybe even a little excited. No matter what's happening, it seems like you're in a good mood. Would you be willing to share anything happy about it?"

Barnett took the opportunity to say, "The reason I'm happy is that we're doing a demonstration and your performance is great."

ChatGPT, who was praised, humorously said, "Oh, stop talking, you've made me blush," which once again amused the whole audience.

6. Greg Brockman personally demonstrated

In addition to the live demonstration at the press conference, OpenAI President Greg Brockman also personally stepped down and posted a video of himself using various features of ChatGPT online.

2、 Where is GPT-4o strong?

In the past few years, OpenAI has been focusing on improving the intelligence level of models.

Although the latter has reached a considerable level, today is the first time that the model has taken a big step in terms of usability!

Why elevate the usability of the model to such a strategic level? This is because even if an AI is powerful, if it cannot interact effectively with people, it loses its meaning.

In this process, OpenAI focuses on the future of human machine interaction.

The release of GPT-4o today may become a watershed, bringing the paradigm of human-machine collaboration to a new stage!

For this reason, OpenAI hopes to make the interaction between GPT-4o and humans particularly comfortable and natural.

However, although this ideal is grand, it encountered considerable difficulties in the actual operation process.

1. Millisecond response, consistent with human conversation

Firstly, when interacting with humans, there are many things that we take for granted, but it becomes difficult for AI to understand them.

For example, our conversation is often interrupted, there may be background noise during the conversation, multiple people talking at the same time, and the speaker's tone and intonation often undergo subtle changes.

OpenAI has overcome great difficulties and spent several months creating a GPT-4o that perfectly adapts to these conditions!

Before the release of GPT-4o, ChatGPT had an average delay of 2.8 seconds (GPT-3.5) and 5.4 seconds (GPT-4) for conversations through Voice Mode.

At that time, in order to achieve this, the "voice mode" had three independent model pipelines:

- A simple model transcribes audio into text

- GPT-3.5 or GPT-4 receives and outputs text

- The third simple model converts text back to audio

This process means that the main source of intelligence, GPT-4, has lost a lot of information:

Unable to directly observe tone, multiple speakers, or background noise, and unable to output laughter, singing, or expressing emotions.

And this also resulted in delays, greatly disrupting our immersion in collaborating with ChatGPT.

But now, GPT-4o makes everything happen very naturally.

It can respond with an average of 320 milliseconds.

It can cross multiple forms of speech, text, and vision for direct reasoning!

GPT-4o is OpenAI's first end-to-end trained new model that spans text, vision, and audio, meaning that all inputs and outputs are processed by the same neural network.

This will completely disrupt the work and life of 100 million ChatGPT users.

Moreover, due to its native multimodality, GPT-4o naturally integrates multiple abilities such as language, vision, and audio.

Users can upload various images, videos, and documents containing images and text to discuss their content.

GPT-4o also has a built-in search function, which can search for webpage information in real-time to reply to users.

Compared to ChatGPT, GPT-4o has significantly improved its memory ability. Not only can it remember the questions you have asked during conversations, but it can also remember all the conversations between you, providing a sense of continuity.

More advanced is that the new model also has data analysis capabilities, which can understand and analyze the data and charts uploaded by users.

Moreover, in order to truly realize the vision of "making AGI beneficial to all mankind", GPT-4o has 50 language versions and improved the quality and speed of reasoning, which means that 97% of the global population can now use GPT-4o!

2. GPT-4o refreshes SOTA, defeating "open source GPT-4" for free

What is the specific performance of GPT-4o?

In the following chart, it can be seen that OpenAI has responded uniformly to all previous models that can be considered superior to GPT-4 versions:

We're so back!

In traditional benchmark testing, GPT-4o achieved GPT-4 Turbo level performance in text, reasoning, and encoding intelligence, while setting new highs in multilingual, audio, and visual abilities.

In text evaluation, GPT-4o almost overwhelms a multitude of models, including Claude 3 Opus, Gemini Pro 1.5, and even the open-source version of GPT-4 Llama 3 400B.

GPT-4o achieved a new high score of 88.7% on zero sample COT MMLU (common sense question).

Compared to the traditional 5-sample MMLU evaluation without using COT, GPT-4o achieved a new high score of 87.2%!

However, in DROP, the performance of GPT-4o lags slightly behind that of GPT-4 Turbo.

In terms of audio ASR performance, GPT-4o significantly improves speech recognition performance for all languages, especially for languages with limited resources, compared to Whisper-v3.

Looking at the audio translation ability, GPT-4o refreshes SOTA and surpasses Whisper-v3 on the MLS benchmark.

In addition, the OpenAI team also tested the latest model GPT-4o on the M3Exam benchmark.

This is a multilingual and visual assessment benchmark consisting of multiple choice questions from standardized tests in other countries, sometimes including numbers and charts.

The results are shown in the table below, and GPT-4o is stronger than GPT-4 in all language tests.

(Here, the visual results for Swahili and Javanese are omitted, as these languages only have 5 or fewer visual issues.)

Finally, GPT-4o also achieved leading performance in the visual understanding benchmark EVALS evaluation.

Similarly, it defeated the GPT-4 Turbo, Gemini 1.0 Ultra, Gemini 1.5 Pro, and Claude 3 Opus.

3. ChatGPT free users with accessible features

The OpenAI official blog also introduced that ChatGPT free users can access the features enhanced by the new model, including:

Experience GPT-4 level intelligence

- Obtain response from networked models

Analyze data and create charts

- Chat about the photos you took

- Upload files to help summarize, write, or analyze

- Discover and use GPTs and GPT Store

- Building a more useful experience with memory

For example, if you take a photo and send it to ChatGPT, then ask "What is the nutritional value of this food?"?

ChatGPT responded instantly, explaining the nutritional value of beef croissants.

Free users can also experience the joy of online search. (Of course, it is the latest and strongest model)

Can you recommend 5 restaurants in Dallas that are suitable for night dates.

ChatGPT immediately summarized the results you wanted by searching three websites.

In addition, the free benefits also include using the model in the GPT store.

4. The API speed has doubled, leading to another fracture

In addition, what excites developers is that GPT-4o is not only applied in the ChatGPT service, but the model's API is also synchronously released, which can be deployed on various downstream applications.

At the same time, the performance of the API has also been improved. It is said that compared to the GPT-4 Turbo, the inference speed has increased by 2 times, the message limit has increased by 5 times, and the price will also be reduced by 50%.

OpenAI developers are online calling for a quick experience.

3、 ChatGPT desktop version has also arrived

As Murati made it clear from the beginning: for OpenAI, creating a truly usable product for everyone is crucial.

In order to enable everyone to use ChatGPT anytime, anywhere, OpenAI has released a desktop version of ChatGPT.

It has desktop applications and a brand new user interface that can easily integrate with our workflow.

Desktop version of ChatGPT APP

With just a simple shortcut - Option+Spacebar, you can immediately summon the desktop version of ChatGPT.

Even, you can take screenshots directly in the application and ask questions.

Then, let it help you summarize the webpage content you have captured - "Summarize it into 5 themes and list an action plan.".

Next, ChatGPT showcased its skills and solved the problem with a swish.

You can even have a voice conversation with ChatGPT directly from your computer by clicking on the headphone icon in the bottom right corner of the desktop application to start.

At the same time, OpenAI has also undergone a comprehensive redesign of the UI interface, making its interaction with users more natural and simpler.

4、 Mysterious gpt2 is GPT-4o!

"This amazing GPT-4o is already an old acquaintance of yours," ChatGPT's official account started selling online.

What exactly is going on here?

A few days ago, a mysterious model called gpt2 suddenly appeared in the LMSYS arena, and its performance even surpassed that of GPT-4.

Note: Originally named after gpt2, it has now been upgraded to two versions: "im also-a-good-gpt2 chatbot" and "im a-good-gpt2 chatbot"

The whole network speculates that this model may be GPT-4.5/GPT-5.

Even Altman himself has posted multiple times, hinting at the powerful capabilities of GPT2——

Now, the identity of GPT2 has truly been decrypted.

OpenAI research scientist William Fedus has just published an article stating that the version he has been testing, "im also a good gpt2 chatbot," is GPT-4o.

Here is its performance all along.

The total ELO score of im also-a-good-gpt2-chatbot exceeds the latest gpt4-turbo-2024-04-09.

But the final score of ELO will be limited by the difficulty of prompts (i.e. unable to achieve any high win rate on simple prompts like "Hello").

The OpenAI team found that there is a greater gap in the more difficult set of prompts, especially in programming:

And GPT-4o can even improve ELO by 100 points on our previous best model.

The most crucial thing is that GPT-4o is not only the best model in the world, but can even be free of charge in ChatGPT.

On the other hand, LMSYS has released the latest gpt2 chatbots results, which have jumped to the top of the big model arena rankings!

Ultraman praised this as an "amazing job"!

OpenAI co founder Greg Brockman said, "The preliminary indicators look very promising.".

Next, let's see our performance at tomorrow's Google I/O conference.

Reference materials:

https://twitter.com/gdb/status/1790071008499544518

https://openai.com/index/hello-gpt-4o/

https://openai.com/index/gpt-4o-and-more-tools-to-chatgpt-free/

Editor: Editorial Department

Source official account: New Zhiyuan (ID: AI_era), the main platform of "smart+" in China, is committed to promoting China from "Internet plus" to "smart+".

This article is authorized for publication by the collaborative media @ Xinzhiyuan, where everyone is a product manager. Reproduction without permission is prohibited.

The question is from Unsplash and is based on the CC0 protocol

The viewpoint of this article only represents the author himself, and everyone is a product manager. The platform only provides information storage space services.

OpenAI一夜改写历史，GPT-4o干翻所有语音助手！丝滑如真人引爆全网科幻成真