Popular in Formal Sciences

Formal Sciences

Diameter rigidity for Kähler manifolds with positive bisectional curvature

Indian Institute of Science Ved Datar

Formal Sciences

Rank-Based Multivariate Sarmanov for Modeling Dependence

McMaster University Anas Abdallah

Formal Sciences

The Hybrid High-Order Method for Polytopal Meshes

Monash University

Formal Sciences

Financial Informatics: An Information-Based Approach to Asset Pricing

University of Surrey

Formal Sciences

Multiplicative orders of Gauss periods and the arithmetic of real quadratic fields

University of Newcastle

Formal Sciences

Some Numerical Methods for the Hele-Shaw Equations

University of Massachusetts at Amherst

Formal Sciences

Geometric stabilisation via p-adic integration

University of Toronto

Creativity Inspired Generative Zero-Shot Learning

View more insights in Sciences - Computer Science - Formal Sciences

View more figures from King Abdullah University of Science and Technology

Next in Computer Science

Software Testing of Generative AI Systems: Challenges and Opportunities

Aldeida Aleti | Monash University

Increasing learner interactions with E-learning systems

Ali Darejeh | UNSW Sydney

Zero-shot learning (ZSL) aims at understanding unseen categories with no training examples from class-level descriptions. To improve the discriminative power of ZSL, Mohamed Elhoseiny discusses the visual learning process of unseen categories with inspiration from the psychology of human creativity for producing novel art.

Read the Study

Image courtesy of interviewee. December 8, 2022

Log-in or Sign-up to Faculti
Currently viewing this subject insight as a guest. You have insight(s) remaining for this month. Login to view 8000+ figures on the platform.

Generating Workspace...

Export as PDF

This video discusses the concept of recognizing unseen visual classes through linguistic descriptions, analogous to how a child can identify a bird based on a verbal description. This video approach can enhance public engagement with wildlife observation and contribute to more accurate data on endangered species.

It outlines a method called "creative AI," which generates new visual stimuli based on unseen classes, improving the distinguishing capabilities of machine learning models. The narrator introduces several iterations of this method, starting with Season version 1, which boosts the model’s ability to recognize unseen classes by encouraging creative visual generation.

Each subsequent version, including Season 2 and Season 3, improves on this idea, with Season 3 utilizing a "random walk" algorithm to ensure diversity in the unseen classes generated. This video highlights how these models can adapt to changing conditions over time, similar to human learning, emphasizing that creativity and imagination are crucial for both human intelligence and advancements in machine learning.

Ultimately, the work advocates using imaginative signals in training AI to enhance its capacity to distinguish between familiar and unfamiliar visual stimuli, drawing parallels to human cognitive processes.

Language:

Export as PDF

X-TIMESTAMP-MAP:LOCAL=00:00:00.000,MPEGTS=900000

00:06.100 --> 00:09.800
So here a short learning is the task of recognizing

00:09.800 --> 00:12.700
unseen visual classes

00:12.700 --> 00:15.700
by just providing a

00:15.700 --> 00:18.100
language description describing how they look

00:18.100 --> 00:21.800
like for example, a kid can can recognize

00:21.800 --> 00:24.600
announcing bird species by describing

00:24.600 --> 00:27.100
it to him or her

00:27.100 --> 00:31.000
for example, if I tell the kid Market open

00:30.200 --> 00:33.400
is a bird species. It has an orange Peak and

00:33.400 --> 00:37.100
the plum is dark about who I below he was

00:36.100 --> 00:40.000
tattoo. Imagine how this bird me look like and if

00:39.900 --> 00:42.200
he or she encountered it

00:42.200 --> 00:45.400
in real life, you'll be able to point point that hey this is a bird

00:45.400 --> 00:48.800
that we just talked about earlier. And this imagine

00:48.800 --> 00:51.900
that this is something that can be helpful to bring

00:51.900 --> 00:54.100
with within the is

00:54.100 --> 00:58.200
system tool that loves the environment. You can

00:58.200 --> 01:01.500
bring a Wikipedia article where the kid can

01:01.500 --> 01:04.600
learn more about these species maybe important were the

01:04.600 --> 01:05.800
kid have seen this species.

01:06.100 --> 01:09.600
And this will make much more people since

01:09.600 --> 01:12.700
the kid was able to identify the species maybe many

01:12.700 --> 01:16.300
more people will be able to recognize and identifying fine-grained

01:15.300 --> 01:18.500
speeches and count them

01:18.500 --> 01:22.000
and hence this information might help decision makers

01:21.200 --> 01:25.000
to know which species are endangered because

01:24.100 --> 01:27.400
we have more accurate statistics about how many we

01:27.400 --> 01:30.600
have for them and everybody can then start to

01:30.600 --> 01:33.200
contribute right if you have

01:33.200 --> 01:37.100
this systems as well instead of it's looking at

01:37.100 --> 01:40.900
it as a tool that makes makes access or

01:40.900 --> 01:43.500
recognition of species much more accessible for everyone.

01:43.500 --> 01:46.500
It can also answer this

01:46.500 --> 01:49.600
question of having models that can tell

01:49.600 --> 01:53.000
that hey this is a species that I can look at.

01:52.300 --> 01:55.400
This possibly has has not been

01:55.400 --> 01:58.700
seen before or maybe could be a new species and maybe

01:58.700 --> 02:01.600
this will because we can host or

02:01.600 --> 02:05.300
both cameras in the ocean on enforce and

02:06.100 --> 02:09.400
stream of information can be continually processed by an AI

02:09.400 --> 02:12.600
system. We might not have many people to go and look

02:12.600 --> 02:15.100
at every single one, right?

02:16.300 --> 02:20.200
So when we have system that can tell hey, this is probably an animal

02:19.200 --> 02:22.100
that I'm not I haven't seen

02:22.100 --> 02:25.900
before this may encourage researchers to to

02:25.900 --> 02:28.300
possibly look in more into that and maybe this will

02:28.300 --> 02:33.900
eventually mainly technology be further developed accelerate discovery

02:32.900 --> 02:35.800
of new species. This is

02:35.800 --> 02:38.900
in part because we have more than 8.7

02:38.900 --> 02:41.700
million species in

02:41.700 --> 02:43.300
our palette planet and actually

02:44.800 --> 02:47.600
Some of these species might might not

02:47.600 --> 02:50.900
be even this discovered before

02:50.900 --> 02:53.300
you go extend because it's a large

02:53.300 --> 02:57.000
scale and not so many people work on actively looking

02:56.800 --> 02:59.500
enforced to discover new species.

02:59.500 --> 03:03.700
So this is what's your planning is seasonal stands

03:03.700 --> 03:06.700
for creativity in this part zero short learning and

03:06.700 --> 03:09.400
this is a method for

03:09.400 --> 03:12.500
organizing unseen classes that inspired from

03:12.500 --> 03:15.200
creative AI.

03:15.900 --> 03:18.600
In in originally creative AI

03:18.600 --> 03:21.500
is the is an approach

03:21.500 --> 03:24.900
to produce creative products, right

03:24.900 --> 03:27.100
the novel creative products like art, for example,

03:27.100 --> 03:30.400
I want to give an access to our authoritage of

03:30.400 --> 03:33.500
Van Gogh and existing art

03:33.500 --> 03:36.400
style like like abstract exceptionism.

03:37.900 --> 03:42.300
Abstract art cubism

03:40.700 --> 03:43.500
Etc. I want

03:43.500 --> 03:46.800
to encourage a machine learning model to produce possibly

03:46.800 --> 03:50.000
new Styles. And that was that that's

03:49.100 --> 03:52.600
that's form of creative AI so

03:52.600 --> 03:55.800
creative eyes about creating an unseen visual

03:55.800 --> 03:58.500
stimuli that may hopefully construct an emotional

03:58.500 --> 03:59.700
experience for for people.

04:01.500 --> 04:04.700
Creative unseen is basically what is meant

04:04.700 --> 04:08.100
what what is meant by that here is to have

04:07.100 --> 04:10.100
the model.

04:10.800 --> 04:13.300
To increase its ability to

04:13.300 --> 04:16.500
distinguish unseen classes are very

04:16.500 --> 04:20.200
similar from scene ones by accurativity

04:19.200 --> 04:22.300
inspired teaching signal.

04:22.900 --> 04:25.400
How do we do that? We have some

04:25.400 --> 04:28.400
seeing classes. Let's say this practical that have an

04:28.400 --> 04:31.100
orange Peak and the blemish with that bird is dark cup of

04:31.100 --> 04:34.200
and white below. Let's assume that for the moment right now. We

04:34.200 --> 04:37.400
moved it to set of scene classes and we want to

04:37.400 --> 04:40.200
we have but we have another class that we

04:40.200 --> 04:43.400
get. We haven't seen yet called crisidoclet. It is

04:43.400 --> 04:46.300
very similar to our kilo. It has an orange Peak and

04:46.300 --> 04:49.400
it has a plumage that they're called White below

04:49.400 --> 04:52.200
but it has this curve with feather that comes from the top of the

04:52.200 --> 04:55.500
head. But this is this cell difference is for us humans

04:55.500 --> 04:58.500
makes it very obvious how we distinguish between both

04:58.500 --> 05:02.000
but from language this perspective. This

05:01.500 --> 05:04.400
can be very challenging from from

05:04.400 --> 05:07.400
machine learning perspective because the language that

05:07.400 --> 05:09.100
describes both words

05:09.900 --> 05:13.000
Will be very similar, right? So since

05:12.300 --> 05:15.900
we don't have access to the language descriptions

05:15.900 --> 05:18.200
of unseen class why we train

05:18.200 --> 05:21.300
will teach the AI model. We need the

05:21.300 --> 05:24.800
the machine learning model to wonder and hallucinate possible

05:24.800 --> 05:27.600
descriptions of unseen classes. And

05:27.600 --> 05:30.100
this is what season is trying to do. So it tries

05:30.100 --> 05:33.400
to explore the creative space or the possible.

05:35.100 --> 05:38.300
Language description of unseen classes and it tries

05:38.300 --> 05:42.400
to push the model till they take this is the generate

05:41.400 --> 05:44.600
visual appearances for

05:44.600 --> 05:47.100
this allocinated unseen class and make sure

05:47.100 --> 05:50.100
that the current model is being is able to

05:50.100 --> 05:54.000
distinguish that from scene classes and hence

05:53.300 --> 05:56.200
this improves the ability here

05:56.200 --> 05:59.100
of the model in later to be

05:59.100 --> 06:02.500
to have an improved ability to distinguish between

06:02.500 --> 06:06.800
this cholesterol class and particular

06:05.800 --> 06:08.600
Class B, despite. We

06:08.600 --> 06:11.700
have a lot in common and only maybe one difference

06:11.700 --> 06:14.100
right? So that's the this is

06:14.100 --> 06:17.900
what season is trying to to do to do

06:17.900 --> 06:20.700
Cecil version one is trying to

06:20.700 --> 06:24.300
increase this distinguishability by encouraging the

06:24.300 --> 06:27.800
visual generations of hallucinated unseen classes to

06:27.800 --> 06:30.300
to be hard to classify from

06:30.300 --> 06:33.700
seeing classes and that improve the

06:33.700 --> 06:34.700
models almost

06:35.100 --> 06:39.500
Double the ability of the model to distinguish these two classes at

06:38.500 --> 06:41.500
this time. If you include this

06:41.500 --> 06:44.300
teaching signal versus you don't

06:44.300 --> 06:47.500
season 2 is is an improved

06:47.500 --> 06:50.900
version of season one. It's it's

06:50.900 --> 06:53.100
more like a journal version of the of the

06:53.100 --> 06:56.300
same work and extension of it where the model

06:56.300 --> 06:59.400
tries to not only encourage the visual generations of

06:59.400 --> 07:02.700
of unseen classes to to

07:02.700 --> 07:06.000
be to to not to

07:05.500 --> 07:08.600
to be hard to classify from seeing ones. But

07:08.600 --> 07:12.200
it also tries to hallucinate multiple unseen classes.

07:12.900 --> 07:15.900
And then tries to encourage the

07:15.900 --> 07:18.600
model to to distinguish them apart

07:18.600 --> 07:21.600
from one another. So this is one anything encourages

07:21.600 --> 07:24.800
the the classifier itself that destroyed to

07:24.800 --> 07:28.200
see distinguish between possible seen

07:27.200 --> 07:30.300
an unseen classes to be informed by

07:30.300 --> 07:34.300
the semantic more informed by semantic the description that

07:33.300 --> 07:36.700
of those classes in

07:36.700 --> 07:39.200
seasonal version one this and

07:39.200 --> 07:42.300
this information was less leverage it.

07:43.900 --> 07:47.500
So that's that's the main difference

07:47.500 --> 07:50.400
and that's improve the performance a little

07:50.400 --> 07:53.200
and also because it have this distinction oblit it

07:53.200 --> 07:56.200
not only to improve its distinguishability to

07:56.200 --> 08:00.600
seeing to to sing classes only but unseen harassmented

07:59.600 --> 08:03.300
unseen classes among each other tizzle shree

08:03.300 --> 08:06.600
has a slide by different approach it

08:06.600 --> 08:09.900
tries to start with let's say anchors of

08:09.900 --> 08:12.900
same classes. Let's say this is a scene class

08:12.900 --> 08:16.100
one seeing class two single class three and

08:15.100 --> 08:18.800
and then Hollister was

08:18.800 --> 08:21.700
hellosmithing generations of unseen

08:21.700 --> 08:24.100
classes. So let's say we have anchors it says

08:24.100 --> 08:28.000
a teen classes over here and they say this are in in green.

08:27.300 --> 08:30.200
Okay, and let's say we generate a

08:30.200 --> 08:33.600
bunch of orange points, but this orange points are visual

08:33.600 --> 08:36.000
generations of unseen classes.

08:36.800 --> 08:39.900
Seasonal version 3 is based on a mechanism

08:39.900 --> 08:41.100
called random walk.

08:41.700 --> 08:44.300
So what we encourage to do is that we start from

08:44.300 --> 08:45.800
with the random walk.

08:46.200 --> 08:49.400
From each sing class go for a

08:49.400 --> 08:52.300
random walks through the if the scene classes is

08:52.300 --> 08:55.300
in greenless, imagine this and then go for a random

08:55.300 --> 08:58.300
walks through the orange points. The orange points are

08:58.300 --> 09:02.100
hallucinated. We are visual generations of hallucinated

09:01.100 --> 09:04.400
unseen classes. And then go for

09:04.400 --> 09:07.200
this orange points for T times

09:07.200 --> 09:10.300
steps and T is is something that is a

09:10.300 --> 09:13.700
design parameter. We use T is equal to 3, but

09:13.700 --> 09:14.300
this can change.

09:15.200 --> 09:18.700
And then encourage your language The Landing probability to

09:18.700 --> 09:21.500
go back to over unseen classes

09:21.500 --> 09:23.800
to be uniform meaning.

09:24.700 --> 09:27.000
No matter which class you start from if you

09:27.300 --> 09:30.900
go through a random walk through hallucinated generations of astroneated

09:30.900 --> 09:33.400
visual generations of unseen classic. It has

09:33.400 --> 09:37.000
to be hard for you to know which scene class

09:36.300 --> 09:39.100
you started from and you can think

09:39.100 --> 09:42.400
of this as a as a deviation signal we

09:42.400 --> 09:45.200
want generations of unseen class intuitively.

09:46.200 --> 09:49.000
To be far from seen ones. So this is

09:49.500 --> 09:52.200
used as a as a deviation signal that

09:52.200 --> 09:56.600
helps separate sink losses from unseen

09:55.600 --> 09:58.300
classes. And hence. The models will

09:58.300 --> 10:02.000
have an improved ability to distinguish between them

10:01.900 --> 10:04.700
what I find interesting in

10:04.700 --> 10:07.300
this learning signal is that there is

10:07.300 --> 10:10.600
a communication you can't think of this random walk

10:10.600 --> 10:14.300
is happening between examples of different of

10:13.300 --> 10:16.400
unseen classes and and

10:16.400 --> 10:19.500
this means that these examples are message passing

10:19.500 --> 10:22.400
information to another and all your

10:22.400 --> 10:26.100
version of Cecil. We encourage every every generation

10:25.100 --> 10:28.200
separately to be

10:28.200 --> 10:32.100
to be hard to classified from from unseen from

10:31.100 --> 10:34.500
seen ones. This is a very example watch

10:34.500 --> 10:37.700
the different visual Generations.

10:37.700 --> 10:40.900
Do not message past one another but in

10:40.900 --> 10:44.100
season 3 these generated examples

10:43.100 --> 10:45.800
of possible unseen classes.

10:46.700 --> 10:50.000
Message pass to one another to collectively

10:49.900 --> 10:53.000
deviate and be different from seeing glasses

10:52.700 --> 10:55.800
and this this message encourage it

10:55.800 --> 10:58.400
to be also more diverse. And hence. It will

10:58.400 --> 11:01.600
improve the generalization a bit and this is what we observe it in

11:01.600 --> 11:02.100
the results.

11:02.700 --> 11:05.800
And one other also different perspective is

11:05.800 --> 11:08.300
that we also applied seasonal version

11:08.300 --> 11:11.400
3 in in a version that we called

11:11.400 --> 11:13.700
continual zero short planning.

11:14.300 --> 11:17.400
What is continual zero short learning? It is a version

11:17.400 --> 11:21.500
of zero short learning worthy the distribution

11:20.500 --> 11:23.600
of scene an unseen classes

11:23.600 --> 11:26.500
changes over time. It's similar to how we learn in

11:26.500 --> 11:29.000
life. Right? So the classes the visual classes that

11:29.100 --> 11:32.500
you have seen, you know today could be different from the visual classes

11:32.500 --> 11:35.100
that you have seen a week from

11:35.100 --> 11:38.900
now, right? So basically that then

11:38.900 --> 11:42.000
that the distribution of scene and unseen classes changes

11:41.100 --> 11:45.000
over time. So in seasonal version 3, we also evaluate

11:44.300 --> 11:47.400
the method on a setting where

11:47.400 --> 11:50.600
they were the we incrementally gain knowledge

11:50.600 --> 11:54.600
and the secretion of scene of seeing

11:54.600 --> 11:57.300
classes changes and we try to see how

11:57.300 --> 12:00.800
the models ability to incrementally recognize I'm

12:00.800 --> 12:02.500
seeing glasses it changes over time.

12:03.300 --> 12:07.300
the major conclusion is that imagination is

12:06.300 --> 12:10.300
a key element of our intelligence

12:09.300 --> 12:11.200
and

12:12.500 --> 12:15.300
What is all about creativity in a

12:15.300 --> 12:18.200
spot zero short planning in all its version? What is the

12:18.200 --> 12:21.600
key element of it? Is that imaginative learning signals

12:21.600 --> 12:24.300
when we inject it to machine learning models.

12:24.300 --> 12:27.600
It will help the model to have an improve

12:27.600 --> 12:30.800
the ability to distinguish what it

12:30.800 --> 12:33.200
has seen so far from what it to

12:33.200 --> 12:36.300
be seen. So what I'm trying to say to all

12:36.300 --> 12:40.000
bring to the table, is that reflecting on

12:39.400 --> 12:42.900
our amazing ability to imagine things and

12:42.900 --> 12:45.300
and and from letting machine learning

12:45.300 --> 12:48.800
techniques that are inspired by that by these

12:48.800 --> 12:51.900
ability seems to be helpful to accomplish

12:51.900 --> 12:53.600
some useful machine learning classes.

Faculti AI can make mistakes

Research

Policy

Analysis