THESIS PROJECT



GENERAL AREA OF INTEREST:
Data Science & AI
in creative industries contexts
CreaTech


RESEARCH TOPIC:
I want to research ...
... Human-in-the-Loop
... Multimodality


PROBLEM:
The problem relevant to AI and data science in the creative industries is ...
... lack of diversity


REASEARCH QUESTION:



SYSTEM:
The AI system used to research / solve the problem is ...
... a voice assistant
... human-in-the-loop system ... active learning**


METHOLOGY:
Design science (methodology)



THESIS CRITERIA
Real world creative-industries relevant research project that can include the development and presentation of fully functioning creative software, such as websites, interactive systems, virtual reality experiences, algorithms for carrying out audio and visual signal processing, and any relevant area of data science and AI in creative industries contexts.


Design Science Research

  • Identify, critically define, and appropriately interrogate known real-world research questions and problems in data science and AI relevant to creative industries contexts


  • Carry out experimental research, including through practical development of software and/or creative outputs.


  • Select and define appropriate qualitative and quantitative methods for interrogating questions.


  • Present results appropriately, including through the use of software, survey analysis, raw data, graphs, visualisations, tables and charts.


  • Critically evaluate their findings based on experimental and/or historical evidence.


  • Conduct an appropriate ethical review of their research plan in line with relevant ethics policies and requirements.




DATA SCIENCE & AI


PROBLEMS

  1. LACK OF DIVERSITY
    A recent study published by the AI Now Institute of New York University concluded that a “diversity disaster” has resulted in flawed AI systems.
  1. POOR DATA QUALITY
    Poor data quality is a significant part of this problem, as AI applications are only as good as the information they can access. Data quality is essential to getting accurate results from your models.

The 10 most cited AI data sets are riddled with label errors,

  1. SKILL SHORTAGE
    This growing demand for AI also means an increase in the demand for developers of AI tech.

Evidence that there are extensive technical skills in AI in UK research institutions as demonstrated by research publications. less evidence that these skills are being used in applications in creative industries

Not sure how to implement AI into creative industries

Can more familiar interaction or interface will help the user with their learning

Fast developments

has become a continuous problem

affects both employers as well as current and future workforce

  1. INACURACY / BAD AT HUMAN-LEVEL
    Companies today might be heavily boasting about accuracy and all, but the truth is humans can handle everything in a much better way.

WHAT IS efficiency?

  1. NON-TRANSPARANCY
    The more people know about AI, the more trust is built around it, the more they’re going to use it.

CREATIVE INDUSTRIES



Human-in-the-loop (HITL) computing


Human-in-the-loop computing is a machine learning approach to
create hybrid intelligence and achieve accurate and interpretable results in machine learning-based AI models. This framework combining data-driven decisions with human decision-making.


Machine learning
Trying to replicate human abilities

Automation

How you want to use it rather than the data needed

Amplifier

Mismach objections between computer and humans

Simulator

What are we trying to optimise for?

Preference Learning
Expose people objectives. Figure out what their reward functions are :

Grow someones reward function
Good mentor exposes you to additional opportunities. You might have something you currently want but are not aware of other optiions.

Real-tima (Matrix)

Computer Vision

HiTL Reinforcement Learning

HUMANS

Humans are not so good to model

Quality of human judgement


TOPICS


Hybrid intelligence System
Interactive AI System

PROBLEMS


TECHNOLOGY & THEORIES



Immersive technologies


Through augmented intelligence and AI, designers can develop more ways for people to communicate, collaborate, and even create tools for entertainment. Companies are going a step further in optimisation with augmented processes, both within business and development.


Augmented Reality (AR)

Virtual Reality (VR)

Mixed Reality (MR)

Creation???

Interactive Systems

Creativity

creative space


Innovation


Innovation is the practical implementation of ideas that result in the introduction of new goods or services or improvement in offering goods or services. Innovation also implies a value system which seeks to derive a positive outcome from the inventive act.Innovation is creating new value and/or capturing value in a new way. Value is the key word, stressing the difference between innovation and invention.


Problem solving machine
Gathering information on how to solve problems

Space

A four-dimensional space (4D)

Chatbot / Assistant

With the growing number of applications of artificial intelligence the inaccuracy of utilised machine learning algorithms could lead to catastrophic outcomes

Internet of Things

Cybernetics

SECTORS

  • advertising and marketing
  • architecture
  • crafts
  • design
  • fashion
  • film, TV, video, radio and photography
  • IT, software and computer services
  • publishing
  • museums, galleries and libraries
  • music, performing and visual arts

IDEAS


CreaTech (creative technologies)
The interface of tech and the creative industries.


It brings together creative skills and emerging technologies to address the emerging field in which technology enables the creative sector to produce new products, services or experiences – and vice versa.


PROBLEMS

IDEAS

Simulation

movements

Train skills in interactive game environment interface?

Make working experience more appealing to the next generation workforce?

Including the use of interaction techniques to invoke actions and solve ML tasks

With the growing number of applications of artificial intelligence the inaccuracy of utilised machine learning algorithms could lead to catastrophic outcomes

IDEAS

DESIGN

Methods

Bio

Multimodal Design

Soma Design

Place humans in the pipeline

APIs

Human decision-making

CHALLENGES

USEFUL

Build a trusting relationship
between humans and AI

Interfaces
Need interfaces to control the pipeline

Co-Creation / Collaboration

The co-creation of tech products and services to improve ways of generating and experiencing creative innovation

Companies rely on building teams that can work across a range of technical and creative disciplines

Only efficient when humans and AI can communicate with each other, e.g., humans communicating with machines through feedback using reinforcement learning, and machines communicating with humans to facilitate decision making using interface.

Machines are learning from humans, but more importantly, the scholars stress the importance of humans learning from the machines

The combination of human and machine intelligence, is more powerful than each intelligence on its own. The concept of hybrid intelligence highlights the complementary strength of human intelligence and AI that can result in a socio-technical ensemble of both. Consequently, social requirements are now an important part of computing design.


HUMAN IN PIPELINE


Application (APP)

Make operational processes more efficient

What is being Effective?????

Applications (APPs) that can be made using machine learning and mixed reality apparent to the creative industries

Web Browser

Principles

Need to be AI-ready

Design Science Research

Human-Centered Design

"Natural surroundings" to the next generation is not the same as this one

Content Creation

Understandability

Grow your skills (HiTL Reinforcement Learning)

Interactive AI System

AI applications struggle with the open-ended nature of problems that humans deal with on a daily basis

USAGE

  • content creation
  • information analysis
  • content enhancement and post production workflows
  • information extraction and enhancement
  • data compression


  • Making predictions

How to incorporate diversity

Client-designer relationship define the success of the project

An algorithm’s outcome, such as a prediction or cluster, should be made more understandable to humans by being interpreted in an expressive manner. ML algorithms lacking interpretability will be challenging to integrate into real-world decision-making processes, particularly with regard to the European General Data Protection Regulation (GDPR) that empowers individuals to receive an explanation to automated decisions made about them

Applying HITL computing can improve an algorithm’s accuracy significantly and contributes in making the ML outcomes more user- friendly and interpretable

By incorporating human intelligence, the creative and dynamic human mind consisting of life experiences, common sense, beliefs and flexibility will be integrated into ML algo- rithms [11,36], that could in return deal much better with unusual data types

This will make ML models more generalizable as training sets would not require to consist of big data, but rather of fewer data selected by human experts

HITL computing would generate a fair AI by “giving the right value to the knowledge producers” and rewarding tacit knowledge inherited by humans

For semi-supervised learning (data sets containing only a small subset of labelled data), humans can act as quality assurance and cross- check predictions made by ML algorithms

In active learning ap- proaches, humans label data while algorithms being in production, thus the ML model will automatically adjust its parameters.

In reinforcement learning, HITL computing can be a tool to regulate AI behavior and inject ethical values

Data Collected

Even though not possible to fully automate it might still be worth it

Sustainable design can be achieved through authenticity in IS design

The highest objective for a designer in HITL computing should be to build a trusting relationship between humans and AI.

click to edit

Gamification

When students use a model of behavior to gain a better understanding of that behavior, they are doing a simulation.

A model or simulation is only as good as the rules used to create it.

Active Learning
**The ML Active Learning Cycle has six steps:**

  • Training Data. An ML model must have data to train on.
  • Build ML Model. The model is created.
  • Model Predictions. The model makes predictions,
  • Feedback. The model gets feedback on its prediction from human or environmental stimuli.
  • Feedback becomes data. Feedback is submitted back to a data repository.
  • Repeat Step 1. Continue to iterate on this cycle.

Simultaneous Improvements

Evolutionary Generative Adversarial Networks


Multimodal Interaction is a situation where the user is provided with multiple modes for interacting with the system.

Multimodal design is a way to think about and design for experiences that integrate our sensory, physical, and cognitive abilities together.

Multimodal design is an approach that recognizes that the whole human sensory apparatus operates differently in different situations.

The patterns of how these sense, the information we need to understand and operate smoothly, and what we expect to do, all change depending on whether we’re having a conversation at a party, jogging down a path, driving a car, or reading a text come together are called modalities.

Understanding these combinations is essential to designing the next generation of user experiences, like VR, Voice, and sensor-based IoT experiences well.

It can also address some of the ways in which digital experiences have become detrimental to our real world lives.

But it also can improve the way we design experiences for all of the different kinds of products and media being created now.

If something is experienced in several senses it’s judged to be more real, we are more certain about it, and the experience or information is more memorable.

Second, having multiple sensory options is important because one sense might not be working for someone, either temporarily or permanently.

Thinking of designing communication or conversation rather than content as stuff

Physical Data / Information

Active Learning

Triangulating Intelligence

WHY TRY TO MAKE COMPUTERS LIKE HUMANS????

The reason why not successful at creating strong intelligent is because we do not know how we function and we are trying to build computers in the image of the human

Application scalability is the potential for an application to grow over time – being able to efficiently handle more and more requests per minute (RPM). It’s not just a simple tweak you can turn on/off, it’s a long-time process that touches almost every single item in your stack, including both the hardware and software sides of the system.


Database scalability is the ability of a database to handle changing demands by adding/removing resources.

Imitation Learning

Meta-Learning

click to edit

Continuous improvement of machine learning and deep learning models requires people for data labeling, quality control (QC), and modeling.

Automation can speed the development process but it requires people to build, maintain, and monitor exceptions.

People design the processes that integrate an AI solution into an organization.

When AI models fail, people must step in to mitigate risk and resolve problems.

topology

Biophilic design