Please enable JavaScript.
Coggle requires JavaScript to display documents.
Cognitive Sciences Related Knowledge (Artificial Intelligence :bar_chart:…
Cognitive Sciences Related Knowledge
Linguistics
:speaking_head_in_silhouette: :abc: :u7a7a:
Areas of research
Neurolinguistics
Language Development
U-Shape Pattern of Development
Begin with copying/imitating
Learning exceptions of rules
Understand rules of grammar
Language Acquisition
Two-Word Stage
Babbling Stages
One-Word Stage
Cooing stage
Developmental linguistics
Intention seeking
Pattern Finding
Justine Cassell
Language aquisition
Historical linguistics
Grammatical Rules
Morphology
Phonology
Syntax
Semantics
Semantic network
semantic parsing, etc
Richard H. Richens, CLRU
WordNet
Montague grammar
Formal Logic
intertranslatability (Jan Landsbergen)
It must be defined clearly what the correct sentences of the source and target languages are.
Kripke semantics
Prototype theory
Eleanor Rosch
there is fixed or static meaning of the words (Derrida)
we learn concepts not an objective truth but from a subjective construct experience
language arises out of the "grounding of our conceptual systems in shared embodiment and bodily experience" (Lakoff, Johnson "
Philosophy in the Flesh: The embodied mind and its challenge to Western thought
", 1999)
Embodied Cognition
Sapir–Whorf hypothesis
Semantic memory
Collins & Quillian (1969) "Retrieval time from semantic memory"
Characteristics Of Language
Communicative
Dynamic
Structured
Arbitrary
Generative
Zipf’s Law
Sociolinguistics
Ethnolinguistics
Semio
Semiotics
Roland Barthes
Umberto Eco
novels
Semiology
Charles Sanders Peirce
Pragmatisme
abduction
against cartheianism
triad
representamen
interpretant
object
Translation / NLP / Computational Linguistics
Clinical linguistics
Evolutionary linguistics
Philology
study of language in oral and written historical sources; it is the intersection of textual criticism, literary criticism, history, and linguistics.
Views/Approaches
Early Grammarians
Pāṇini Sanskrit morphology
cuneiform clay tablets
Grece and the Stoics
Sibāwayhi arabic linguistics
Irish Sanas Cormaic 'Cormac's Glossary' encyclopedic dictionary
'De vulgari eloquentia' Dante
Structuralism
Ferdinand de Saussure
diachronic
synchronic
syntagme
paradygme
signifiant / signifié
The Prague school
Roman Jakobson
6 language functions
Leonard Bloomfield
Louis Trolle Hjelmslev
Generativism
Noam Chomsky
Functionalism
Cognitive linguistics
George Lakoff
Linguistic Wars
Linguistic turn
continental post structuralism VS logic and analytic philosophy of language
Psychology
:speech_balloon: 🧠 :left_speech_bubble:
Views/Approaches
Functionalism
Structuralism
(1880-1900/60)
People
William Wundt
basic elements of thought
-
Sensations
:basic elements of perception
-
Feelings
:basic elements of emotion
Analytic Introspection
“Observation of one’s own thought processes”
Required training
High confirmation bias
Unreliable and not objective.
Jacques Lacan
Ferdinand de Saussure
Claude Levi-Strauss
What ?
Methods
Limits
Voluntarism
Psychoanalytic Psychology
Associationism
(1880-1920)
People
Edward Lee Thorndike
Hermann Ebbinghaus
What ?
Methods
Limits
Behaviourism
(1920-1960)
What ?
Types of Learning
Operant Conditioning (Skinner)
Punishment
Negative Reinforcement
Positive Reinforcement
Reinforcement
Positive Reinforcement
Negative Reinforcement
Classical Conditioning (Pavlov)
CS
UR
CR
US
Methods
Limits
People
Burrhus Skinner
children learn language through operant conditioning
Children imitate speech they hear
Correct speech is rewarded
Ivan Pavlov
John Watson
Edward Tolman
Rat maze experiment
Karl Lashley
Constructivism
(1923-?)
What ?
Methods
Limits
People
Jean Piaget
Gestalt
(1920-1950)
What ?
Methods
Limits
People
Kurt Koffka
Wolfgang Köhler
Max Wertheimer
Kurt Lewin
Cognitivism
(1950-?)
What ?
Methods
Limits
People
Noam Chomsky
language must be determined by inborn biological programming
Children say things that:
They have never heard and can not be imitating
That are incorrect and have not been rewarded for
Georges Miller
Jerome Bruner
Jerry Fodor
Herbert Simon
Allan Newell
Jean Piaget
Claude Shannon
Warren Weaver
Marvin Minsky
John McCarthy
Connexionism
(1970-?)
What ?
Methods
Limits
People
Frank Rosenblatt (American psychologist)
Warren McCulloch
Walter Pitts
Donald Hebb
David Rumelhart
David Parker
Embodied Cognition
(1990-?)
What ?
a brain inside a body inside an environement
we experience the world by living in it
Linguistics
autopoiesis
Methods
Limits
People
Francisco Varela
Maturana
Evan Thomson
Eleanor Rosch
https://tinyurl.com/courants-psychocog
Evolutionnary psychology
Theories
Inclusing Fitness (William D. Hamilton)
Theory of Natural selection (Charles Darwin)
Nativism
Jerry Fodor
Noam Chomsky
Stephen Pinker
Criticism
nature VS nuture
Richard Dawkins
Criticism
Social Cognition
Cognitive Psychology
Developmental & Learning Psychology
Psychopathology
Diagnostic and Statistical Manual of Mental Disorders (DSM5)
Major depressive disorder
Bipolar disorders
Dysthymia
Schizophrenia
Borderline personality disorder
Bulimia nervosa
Phobias
Pyromania
Personality Psychology
Hans Eysenck 3 types
extraversion-introversion
neuroticism
psychoticism
Big 5 personality traits (FFM) ( Lewis Goldberg) (OCEAN)
Openness to experience
Conscientiousness
Extraversion
Agreeableness
Neuroticism
The four Ds
Deviance
Distress
Dysfunction
Danger
Psychanalisis
Lacan
"la femme n'existe pas"
"il n'y a pas de rapport sexuel"
Zizek
"pure ideology"
"and so on" snurff
Freud
"parlez moi de votre relation avec votre mère"
Psychosocio
Cliodynamics
mathematical modeling
History
:classical_building: :amphora: :mantelpiece_clock:
Economics
:chart_with_upwards_trend: :moneybag: :currency_exchange:
Movements
liberalism
Peter Turchin
Quantitative Analysis of Movement: Measuring and Modeling Population Redistribution in Animals and Plants,1998
Historical Dynamics: Why States Rise and Fall, 2003
War and Peace and War: The Rise and Fall of Empires, 2006
Ultrasociety: How 10,000 Years of War Made Humans the Greatest Cooperators on Earth, 2016
articles
Dynamical role of predators in population cycles of a forest insect: an experimental
, 1999
Are lemmings prey or predators?
, 2000
Population Dynamics and Internal Warfare: A Reconsideration
, 2006
Long-term population cycles in human societies
, 2009
War, space, and the evolution of Old World complex societies
, 2013
Ibn Khaldun
The Muqaddimah
Fernand Braudel
Civilisation Matérielle, Économie et Capitalisme
La Méditerranée et le Monde Méditerranéen à l'Epoque de Philippe II
"Longue durée" history
Carl von Clausewitz
Vom Kriege
Philosophy
:books: :thought_balloon: 🤔
Philosophy of Mind
The Mind Body Problem
Dualism
Property Dualism
Substance Dualism
Classical Dualism
Monism
Idealism
Physicalism
Functionalism
Knowledge-Acquisition Problem
Nativism
Associationism
Types of Knowledge
Declarative
Procedural
Rationalism
Empiricism
Consciousness
The Chinese Room Argument
Cartesian Theatre
The Multiple Draft Model of Consciousness
Approaches
Metaphysics
Epistomology
Philosophy of Science
Concepts
Scientific method
Methods - Logic and Reasoning
Scientific Method
Popper (il pue)
Hempel: Paradox of confirmation (paradox of Ravens)
Inductive
Bayes !
Hume
Nelson Goodman Puzzle of Grue/Bleen observed/non observed emeralds
Abductive
Deductive
Hypothetico-deductive model
A priori and a posteriori
Causality
Empirical evidence
Falsifiability
"Objectivity"
Metatheories
Structuralism
Inductionism
Evolutionism
Pragmatism
"Consider the practical effects of the objects of your conception. Then, your conception of those effects is the whole of your conception of the object."
Peirce, C. S. (1878)
Charles Sanders Peirce
William James
John Dewey
Positivism
Reductionism
Determinism
Rationalism
Empiricism
Schools of thought
Contemporary
Analytic
Concepts
Analysis
Supervenience
Theories
Logical positivism
Neurophilosophy / logical behaviorism
Descriptivist theory of names
Emotivism
Functionalism
Quietism
Philosophers
Nick Bostrom
Noam Chomsky
David Chalmers
Daniel Dennett
Jerry Fodor
Karl Popper
John Searle
Bertrand Russell
Ludwig Wittgenstein
Kurt Gödel
Thomas Kuhn
Willard Quine
Saul Kripke
Gottlob Frege
Continental
Theories
Absurdism
Deconstruction
Existentialism
Frankfurt School
German idealism
Hegelianism
Hermeneutics
Phenomenology
Postmodernism
Post-structuralism
Structuralism
Neo-Kantianism
Marxism
Philosophers
Theodor W. Adorno
Louis Althusser
Hannah Arendt
Gaston Bachelard
Alain Badiou
Roland Barthes
Jean Baudrillard
Simone de Beauvoir
Henri Bergson
Albert Camus
Emil Cioran
Guy Debord
Gilles Deleuze
Jacques Derrida
Umberto Eco
Michel Foucault
Félix Guattari
Jürgen Habermas
Friedrich Hegel
Martin Heidegger
Edmund Husserl
Søren Kierkegaard
Julia Kristeva
Jacques Lacan
Emmanuel Levinas
Jean-François Lyotard
Maurice Merleau-Ponty
Karl Marx
Friedrich Nietzsche
Jean-Paul Sartre
Arthur Schopenhauer
Michel Serres
Gilbert Simondon
Peter Sloterdijk
Slavoj Žižek
Concepts
Existence precedes essence
Ideology
Master–slave dialectic
Master–slave morality
Oedipus complex
Ressentiment
Will to power
Class struggle
Dasein
Death of God
Différance
Modern
Early modern
Kantianism
Phenomenon
is "an observable fact or event"
Noumenon
is a posited object or event that exists independently of human sense and/or perception.
Things-in-themselves
would be objects as they are, independent of observation. The concept led to much controversy among philosophers.
René Descartes
Thomas Hobbes
Blaise Pascal
Baruch Spinoza
Wilhelm Leibniz
Isaac Newton
John Locke
George Berkeley
David Hume
Niccolò Machiavelli
Martin Luther
John Calvin
Michel de Montaigne
Francis Bacon
Immanuel Kant
Montesquieu
Adam Smith
Jean-Jacques Rousseau
Cartesianism
Late modern
Friedrich Hegel
German idealism
Gottlieb Fichte
Joseph Schelling
Jena Romanticism
Arthur Schopenhauer
Friedrich Nietzsche
Charles Darwin
Auguste Comte
John Stuart Mill
Karl Marx
George Boole
Gottlob Frege
Henry Sidgwick
Charles Sanders Peirce
William James
Hegelianism
Marxism
Ancient
Chinese
Greco-Roman
Indian
Persian
Medieval
European
East Asian
Indian
Islamic
Jewish
Philosophy of Language
Linguistic relativity
Strong
Linguistic determinism
Pros:
George Orwell's 1984 Newspeak
Eskimo snow terms
color terminology
Pirahã language
Hopi concept of time (studied by Whorf )
la tribu qui situe les trucs en fonction des points cardinaux
Cons:
Lexical differences
Structural differences
The world is experienced differently by speakers of different languages. Language is causally linked to these cognitive differences (Lenneberg, Brown,1954)
Criticism of several examples
Humans think not in individual languages, but in a shared language of thought, in order to translate this "Mentalese" (universal language of thought) into a string of words for the sake of communication
(Steven Pinker influenced by Chomsky)
trendy in the 70s
language
determines
though and decisions
Weak
language
influence
though and decisions
Linguistic influence
Sapir–Whorf hypothesis
Edward Sapir and Benjamin Lee Whorf never co-authored any works. Sapir made an observation that his student Whorf pushed further to make a theory
KEY IDEAS (of Sapir)
1) laguage gives acess to a world with less restrictions than the real world
2) the real world cannot be experienced directly, language influence our experience and we cannot abstract ourself from it
3) Hypothesis: if our language influence our world perception, maybe different languages influence it differently
(Sapir,
Language
, 1933)
KEY IDEAS (of Whorf)
He observe the Hopi language and conclude than
Indo-europeans see a 'tridimentional static and infinite space and a unidimentional cinetic time always moving
Hopi have a distinction between objective (senses) and subjective (futur, imagination, emotions) and that can be simplified as "experienced reality" vs "non experienced reality" and no notion of time
People who interpreted their work (of Sapir and Whorf), drew from these observations that some populations are "more evolved" than others
TO KNOW
Future tense in indo european languages is 'recent'
There are a lot of tribes that use a cyclic time representation bc it correlates with their activities close to nature
Study on calling schizophrenia "schizophrenia"
Results: people are more up to have a meeting with a 'mentally blablabla person' than w a 'shizophrenic' BUT also these people who made this choice had clichés in mind about what is schizophrenia.
CCL: language will change your behaviour but this behaviour is initally activated by your thoughts
(Penn, Drummont, 2001)
to rewrite because it's a mess
Deconstruction
Metaphysics
Ontology
Aesthetics
Ethics
Epistemology
Logic
Philosophy of Mathematics
Structuralism
abstract structuralism
ante rem
("before the thing")
(réalisme en linguistique ?)
Logicism
Platonism
Structures are held to have a real but abstract and immaterial existence
eliminative structuralism
post rem
("after the thing")
(nominalisme en linguistique ?)
Nominalism
modal structuralism
in re
("in the thing")
(conceptualisme en linguistique ?)
Aristotelian realism
Structures are held to exist inasmuch as some concrete system exemplifies them
No free lunch theorem
Bayesian probability
Ethics in mathematics
Formalism
Hume's principle
Ludwig Wittgenstein's philosophy of mathematics
Reality
Logic
Deductive logic
if P--> Q
P, therefore Q
Infinity
Anthropology
📿 :silhouettes: :memo:
Behavioural Economics
Loss Aversion and Framing Affect
Mental Accounting
Loss Aversion
Sunk-Cost Fallacy
The Ultimatum Game
Endowment Effect
Cognitive Anthropology
Evolutionary Psychology
Evolution and Judgment under Uncertainty
Fallacies
Gambler's Fallacy
Base Rate Fallacy
Conjunction Fallacy
Heuristics
Evolution and Natural Selection
Variation
Selection
Inheritance
Evolution of Logic and Reasoning
The Watson Selection task
Hard Version
Easy Version
Evolution and AI
Evolution and Language
Evolution and Sex Differences
Comparative Cognition
Animal Problem Solving
Transitive Interference
Comparative Neuroscience
Human brains compared to other animal brains in terms of size, structure, and cognition
Cephalization Index
Object Permanence
Study of Human Origin through evolution
Critique
too ethnocentred
too much "western view"
Ethnology
Ethnography
Linguistic Anthropology
Field
Sociocultural
Biological
Archaeological
Linguistic
Approaches
Structuralism
Claude Levi Strauss
Philippe Descola
Interactionism
Chicago School
Social constructionism
Bruno Latour
"non-modernité" (Nous n'avons jamais été modernes : Essai d'anthropologie symétrique)
Actor-Network theory
study scientific community from an anthropologic point of view
Neuroscience
:rat: :microscope: 🧠
Neurobiology
Neuroanatomy
Neurons
Structures
Myelin Sheath
Soma
Cell Body
Nucleus
Synapse
Dendrites
Axon Terminals
Neural Proces
Cortical Features
Cerebral Hemispheres
Gyrus
Fissure
Triune brain model (MacLean, 1960) Reptilian / Paleo / Neomammalian
Directions in the Nervous System
Posterior "Back"
Medial "Middle"
Ventral "Bottom"
Dorsal "top"
Lateral "Side"
Anterior "Front"
Limbic System
Diencephalic structures
Hypothalamus (many functions, hormones, emotions, homeostasia, sleep, circadian rhythms)
Mammillary bodies (episodic memory)
Anterior nuclei of thalamus (learning, episodic memory
Lateral nuclei
Pulvinar
other structures
Medial nuclei
Anterior nuclei
geniculate nuclei
Cortical areas
Hippocampus (memory consolidation)
parahippocampal gyrus (encoding and retrieval)
Limbic lobe
Piriform cortex (olfactory track)
Entorhinal cortex (time perception, memory)
Fornix (recall, recognition)
Subcortical area
Septal area / nuclei (theta waves: meditation, relaxation, focus, mental imagery)
Amygdala (memory, decisions, emotions)
basolateral complex
central nucleus
cortical nucleus
Nucleus accumbens (motivation, reinforcement)
Shell (reward, positive reinforcement)
Core (slow-wave sleep)
Papez circuit
Visual system
Visual paths
magnocellular / pavocellular pathway
retino cortical pathway
retino tectal pathway / colliculus superior (movement)
dorsal / ventral pathway
pulvinar
retina
chiasmal optic nerve
geniculo striate pathway / occipital cortex (striate) / forms, colors, patterns, recognition
Retina cells
ganglionnaire cells
Parasol / M / W / gamma / 10% / contrasts
Midget / P / X / beta / 80% / details, color (in fovea)
Bistratified / K / Y / alpha / 10% / movement (in periphery)
melanopsin ganglion cells / pupillary reflex, circadian rythms
lateral inibition
amacrine cells
bipolar neuron
depolarized ON / less glutamate = less inhibited
hyperpolarized OFF / less glutamate = less activated
photoreceptors
Rod cells (luminosity)
Cone cells (short wave, middle wave, long wave) / colors
Eye
Subcortical Features
Basal ganglia
Striatum
caudate nucleus
putamen
Pallidum
globus pallidus (GPi, GPe)
Substantia nigra
Neurophysiology
Neurochemestry
Neuropsychology
Executive Functions
Models
Supervisory attentional system (SAS) by Norman et Shallice (1980)
Miyake (2000)
Diamond (2013)
Luria (1966)
Functions
Divided attention (Kahneman)
Focused attention
Inhibition
Flexibility
Inductions
Deductions
Planification
Tests
Inhibition
Go-NoGo (TEA)
Hayling (Burgess & Shallice, 1996)
N-Back test
Stroop
Flexibility
Trail Making Test
Graphical fluency (Ruff et al., 1987)
unusual objects (Eslinger & Grattan, 1993)
graphical series
gestual series
Categorisation (inductions)
Wisconsin Card Sorting Test (Nelson, 1976; Heaton, 1981)
California Sorting Test (Delis et al., 1992)
Deductions
Brixton Test (Burgess & Shallice, 1996)
Planification
London Tower
Ecological tests (with daily life)
Behavioural Assessment of the Dysexecutive Syndrome (BADS) (Wilson et al, 1996)
Party Task (Chalmers et Lawrence, 1993)
Travel Task (Miotto et Morris, 1998)
6 elements task (Shallice et Burgess, 1991)
Iowa Scale
DEX Survey (BADS) (Wilson, Alderman, Burgess, Emslie & Evans, 1996)
Memory
Categories
LTM
Long Term Memory (t: days, months,years)
Declarative (explicit / conscious)
Semantic (concepts)
Episodic (events)
Non-Declarative (implicit / unconscious)
Procedural (skillz, actions)
Priming (identification of objects and words)
Emotional
classical & operant conditioning
Somatically / Sensory (t:1 sec)
STM
Short Term Memory (t:1 min) / 7+-2 (G. Miller)
Abstract (central executive)
Phonological
Visuo spacial
Tests
IQ
STM: "empan" (fr) verbal / visuospacial direct memory span
Working Memory: "empan" (fr) verbal / visuospacial indirect memory span
Anterograd Memory: verbal / visuospacial recall
Retrograd Memory
Semantic: Supermarket fluency (Quillian & Collins, 1969)
Autobiographic
Semantic: personal identity info recall
Episodic: personal habits info recall
TEMPau
Implicit
perceptual
conceptual
Procedurale: Hanoi tower / London tower
Pathologies
Amnesias
Organic
Permanent
Evolutive
Dementia
5 more items...
4A
4 more items...
Alteration LTM/STM
Stable
Focal (prosopagnosia)
Global
3 more items...
Transitory
Symptomatic
Idiopathic
Psychogene (fonctional)
Dissociative amnesia (after a traumatic experience)
Dissociative fugue state
focal retrograd amnesia
Models
Atkinson & Shiffrin information processing and memory model (1968)
Attention
Conciousness
Arousal ("vigilance")
Cannon–Bard
James–Lange
Decision Making
Natural Language
Learning
Motor coordination
Perception
Planning
Problem Solving
Thoughts
Ressources
Neuroscience: Historical Perspectives
Localisation and neuroanatomy
17th Century: Cartesian Legacy
Descartes
: brain and behaviour mediated by a mechanical theory that nerves are hollow with fluid flowing through them -
'Balloonist' Theory
. Believed mind located in pineal gland.
Dualism
mind and body are separate!
Pineal gland: small endocrine gland in brain between two hemispheres (central so logical to place mind here), it produces melatonin.
:no_entry:Ghost in the machine:
Cartesian Gap
how could something immaterial influence something material?
Willis
: post-mortem examinations, concluded that psychological attributes are heavily reliant on the
cortex
! Dualism but believed the point of contact between mind and body is the cortex. :no_entry:Cartesian Gap
16th century: Non-Ventricularist
rejected previous theory! Evidence that all mammals have the same anatomical organisation as humans but do not have the same intellect. Believed that rather than the soul, it was animal spirits which reside in ventricle and follow nerves to organs.
13th Century: the mind and brain are related but seperate
Magnus
brain is the centre of mental activity: sensation, rationing, memory. Mind and brain inextricably linked, but mind immaterial and resides in ventricles.
19th century: Advances in microscopy, improved histology, debate about nervous system organisation,
Fritsch & Hitzig
15th century: the Rebirth of Neuroanatomy!
Da Vinci
: anatomical contribution of structure of the ventricles. Believed soul resides in ventricles rather than tissue because it is incorporeal.
18th century was useless!
The Neuron Doctrine
Ramon y Cajal
neuroanatomist:
The Neuron Doctrine!
: acceptance of the neuron as anatomical and functional unit of the nervous system
Investigated nervous conduction: direction of impulses and idea of synapses.
Won Nobel Prize (against Golgi!)
1887: new staining method! fixative and silver nitrate twice to get deeper staining... lead to observation in chicken cerebellum samples that dendrites reside at axon endings.
"each cell is a totally autonomous physiological canton"
Cell theory
"the relationship between nerve cells was not one of continuity but rather of contiguity"
--> not continuous, just in close proximity
Golgi staining
the silver nitrate method invented by
Golgi
in 1873
won Nobel Prize (against Cajal!)
"Rete Nervose Diffusa" in Italian which literally translates to widespread nerve network
Golgi believed his observations of cerebellar tissue etc were evidence of a diffuse nerve network hence supporting
reticulum
approach - as he observed continuous network rather than discrete nerves.
a more holistic approach
Reticularists
v
Cell Theorists
Both
: use the same methods but came to different conclusions :warning:low magnification and poor resolution of microscopes available at the time
R
: believe NS is a large network of tissue (reticulum), formed by fused nerve cells. "The whole brain is a syncytium" meaning a large, cell-like structure.
Von Gerlach
C
: believe that NS consists of distinct nerve cells. Controversial theory that autonomous life should be attributed to cells.
Virchow
"omnis cellula e cellula" cells can only multiply from cells.
Reasons for delay: why did it take so long to realise this?
belief of cell continuity was necessary for cell interaction at that point of theoretical knowledge: i.e. did not make sense to scientists that discrete units could interact- they must be connected!
Technical advancement of histological prep and microscope accuracy
nerve cells/ NS have very complex structure!
1950s: Electron microscopy techniques:
Bennett
: Reaffirmed Cajal's proposal that there is a gap (synapse) between all nerve endings, even when greatly intimate.
advancing technology aided advancement in understanding of the nervous system and individual nerve cells.
Dendrites, cell body, axons identified.
Main Tenets of the theory
information flows in one direction along the neuron
Neurons are discrete not continuous
The fundamental structure/ unit of the NS is the neuron
Neurons have 3 parts: dendrite, cell body, axon
1820
Purkinje cells
:
Johannes Evangelista Purkinje
: discovery of these cells made him famous, they are among the largest cells in the vertebrate brain
Waldeyer
: coined the term 'neuron' in 1891
Early theories: the mind and body
Galen of Pergamon
:
Encephalocentrism
: anatomical methodology, brain receives sensations and generates understanding.
"apodeictic proofs" provided: brain and spinal cord source nerves. Spinal cord originates from brain
Experiments: severing throat nerves in pig inhibited squealing (but still breathing), concluded that voice comes from brain and not the heart.
Soul: communicates with the body via spirits, through 3 ventricles (brain cavities!) & travel through body via nerves.
Plato
: the brain conveys sensations and once these settle in the mind, knowledge, memory and opinions are produced.
Encephalocentrism
but also cardiocentrism!: 3 species of soul
Epithemetikon: perishable soul, responsible for urges e.g. hunger, passion, located in liver
Thymos: perishable soul, responsible for emotions, located in chest/heart
Logos: immortal, divine, localised in head
Western Medicine
:
Hippocrates
: noted 2 hemispheres of the brain; 'lateralization' and rejected possibility of a 'sacred' cause of epilepsy: said it was a disease of the brain. Believed the brain is the most powerful organ
Encephalocentrism
vs
Cardiocentrism
debate
Where lies the 'seat of human consciousness, sensation and knowledge'? Brain vs heart!
Aristotle
: 3 soul faculties, all reside in heart:
cardiocentrism
Nous: humans, an immaterial soul
Vegetative: plants and some animals
Believed the role of the brain to be: diminish heat of blood generated by heart and generate sleep
Sensitive/motor: animals
Debate continued into the Renaissance!
Homer
:
One overall 'soul' called 'psyche'
: non-localised soul which represents individual life/identity. soul is active/non-silent only in dreams and abandons the body at death
three additional types of 'body soul'
located in chest
Noos: reasoning
Menos: aggression
Thymos: emotion
Egypt!
48 cases of head injuries investigated: noted the importance of the brain and it's role in behaviour but believed the heart was the 'seat of the soul' (mummification)
Natural Philosophers
: systematically reject supernatural and try to understand everything through material mechanisms
e.g.
Anaximenes
who theorised that the origin of human thoughts is air.
Anatomical dissection:
Alcmaeon of Croton
:
Encaphalocentrism
: sensory and cognitive significance of brain lead him to believe it was where consciousness resides & if the brain is disrupted then so are all senses. Also, believed that man has understanding whilst animals only have sensations
Cardiocentrism
: THE HEART IS...: affected by emtion, all animals have one, source of blood, warm, connected to whole body, essential, formed first in foetus, sensitive to pain, central in body!
Localisation: Phrenology to cortical topography
"the first brain map":
Fritsch & Hitzig
: dogs' cortex electrically stimulated resulted in muscular contractions. :check: localization, as findings generalised to say that all functions are linked to specific areas.
19th century
Localization theory
: high degree of specialization of brain areas' functionality
Gall & Spurzheim
Brain Equipotentiality theory
: all parts of the brain have equal significance in functions
1585:
von Grafenberg
: early observations of brain damage - although the tongue was not paralysed the patient could not speak: Same as
Galen
- voice must come from the brain
Anatomy & brain mapping:
Brodmann
: systematic investigation of primates developed somatosensory map: adjacent areas of body represented in adjacent areas of cortex.
Jackson
: Epilepsy: often convulsions are on one side of body only - perhaps because the organic disease is only affecting one hemisphere?... motor cortex organised somatotopically: i.e. hands are most sensitive/wide range of movement so must receive largest representation in cortex.
more research builds bigger picture:
Penfield
Homonculus!
Phrenology
: Principles: Many organs in the brain, responsible for particular skills. Bigger = more powerful. The brain shape is determined by organ development. Skull surface provides accurate index of brain. Can analyse personality using this method. Phrenometer: machine which measures 'bumps' on skull.
but they (Gall & Spurzheim) did make important observations, such as the fact that the brain is folded.
in reality, skull shape has nothing to do with cognition.
Brain mapping: lesions:
Broca
's area & aphasia: left inferior frontal cortex damage thought to be cause of language/speech inhibition.
Against localization:
Flourens
: brain lesions in animals did not cause specific behavioural deficits, hence concluded that brain shares functionality:
Aggregate field theory
Mapping and understanding of 'mind': explain beh in mechanistic terms, dualism -> materialism, reductionism: explain brain in terms of its parts :red_cross: does this really inform cognitive models? are mind and brain separate?
Wernicke
's aphasia: damage to posterior left hemisphere: language understanding rather than speech.
https://coggle.it/diagram/XbQvj0Ub1g6An_AA/t/neuroscience-historical-perspectives
Neuroscience:
Research Contribution to CS
Cognitive Neuroscience:
The Biology of the Mind
Human Brain: The 3 Main Parts
Cerebrum
Its outer folded layer is called
the cerebral cortex.
The surface of the cerebrum is
the cerebral cortex
Cerebral Cortex
FOUR MAJOR AREAS
4 more items...
3 mm thick
Very thin layer
The area is a convoluted pattern
packed into a small space inside the
skull.
Surface of brain = cortex
(i.e. gray matter)
Decision making organ of
the body
Receives messages from
all sensory organs
Initiates all voluntary
actions
Stores all our memories
Where our knowledge of
language resides
Cerebral Hemispheres
Composed of cerebral
hemispheres (left & right),
joined by corpus callosum
Corpus Callosum = large band of fibers that connects the
left and right hemispheres of the cerebral cortex
Contralateral brain
function
Left hemisphere controls right
side of body
Right hemisphere controls left
side of body
The left and right hemisphere
are separate, but connected
The cerebrum is the largest
part of the brain and controls
all conscious thoughts,
experiences, and actions
Divided into right and left
hemispheres, which are joined
by the corpus callosum
Brain Stem
It controls processes basic for
survival, such as heart rate,
breathing, digestion, and
sleep.
It also has its own set of nerves
that send and receive signals to
the face, mouth, tongue, eye
muscles, ears, and balancesensing
vestibular organs
It is the main route of
communication between the
rest of the brain, the spinal
cord, and the nerves that run
throughout the body
Cerebellum
Recent research also
suggests a role in higher
cognitive processes
The cerebellum is the
second largest part of the
brain
It controls posture and
balance.
Human Brain
Approximate length = 15 cm
Consists of approximately 10
billion nerve cells (neurons) and
billions of interconnecting fibers
Approximate weight = 1400 g ( 3
pounds/ 1.4 kg)
Newborn = 350 to 400 g
Most complex organ of the body
Brain: The Nervous System
(Vertebrate)
Neurons
Dendrite
= Receives messages.
Brings information to the cell
body
Neuron connections
This makes the brain the most
complex organ and most complex
structure in this universe
Huge number of interacting
neurons in the brain
Axon
= Sends message. Takes
information away from the cell
body
Soma
: body of the cell.
Contains genetic material
nerve cells, specialized to receive and transmit
information in the nervous system
Communicate through stimulation of electric impulses/signals
Synapse
= a small gap
separating 2 neurons
a link from one neuron to
another
Neurotransmitters
This makes our brain work
determine how we feel, think and
act
Chemical signals that are released
across the synapse so that neurons
can communicate with each other
Brain Research: The Development
Insights from Psychology: Sheperd Ivory Franz
Training of animals in new habit after removal of brain
parts (frontal lobe)
New habits lost but old ones retained
Animals could relearn lost new habits
Generalized to human
Functional restitution or reeducation after brain lesions –
evidence showed that lost function can be recovered (i.e.
functions could not have resided in damaged area alone)
Brain = integrated whole
Challenged theory of localization of function based on
findings on study of experimental animals’ behavior
Late 19th & Early 20th Century
Brain – the organ of the mind
Overall – primary investigative techniques
employed include electrical stimulation or
removal of specific brain areas
Theory of localization of function
David Ferrier
continued experiments on other
species (i.e. monkeys, frogs, guinea pigs) to find
out whether the theory can be generalized
Used the method of ablation (surgical removal) or
also called localized destruction
Extended investigations beyond “motor centers” to
include sensory centers
E.g., vision (perception) and hearing
Insights from Psychology: Karl Lashley
Proposed Principles
Principle of Equipotentiality
Parts of the brain have the
equal potential to fulfill the
function of a damaged part
The intact part of the brain
can carry out the functions
that were lost in the
damaged part. This capacity
varies from one area to
another and from one
function to another.
Principle of Mass Action
When a functional area of
cortex is damaged, its ability to
perform previously mastered
complex functions may be lost
in proportion to the extent
of the injury
Proposed 2 principles to explain his conclusion
Principle of Equipotentiality
Principle of Mass Action
Theory of localization – oversimplified; argued instead that
functions of every center dependent on its relation to rest
of intact nervous system
Based on findings from experiments with white rats which
investigated their learning & intelligence
Intelligence = global function because learning can recur
(relearn)
Late 19th & Early 20th Century
Evidence in support of the theory based on
Fritsch & Hitzig’s
experiments on dogs
Muscle movement stimulated by direct electricity
application to brain
Hughlings-Jackson’s
observation of recurrent
patterns of epileptic seizures in patients -
connection between stimulus event in
specific brain areas and muscular response of
specific body side
Localization of function in the two
hemispheres
Theory of contralateral control (opposite-side)
Late 18th & Early 19th Century
Gall
Conclusion that structure & function were related
Structure of brain & people’s behavior
Phrenology
Emergence of theory of localization
Broca’s Area
Wernicke’s Area
The Neuroscience Approach (Cognitive Neuroscience)
Neuroscientists
Learn more the structures and functions
of the human brain and how they relate
to the behaviors we observe in people:
using language, solving problems, and
remembering
Attempt to explain cognitive
processes in terms of underlying
brain mechanisms
Describe the biological ‘hardware’ upon
which mental ‘software’ supposedly runs
Brain Plasticity
Radical suggestion, radical solution
Hemispherectomy
The brains of children have amazing abilities to
rewire themselves
Since the left side of the body is controlled by the right side of the brain,
the patient would be paralyzed on the left side when
they are awoke
Removal of half of the brain
Surgery to have one side of the brain removed to prevent
seizures.
Someone could be playing and become suddenly rigid
and collapse on the floor
Phantom Limbs
Approximately 5-10% of individuals with an amputation
experience phantom sensations in their amputated limb, and the
majority of the sensations are painful
Sensation that an amputated or missing limb (even an organ, like
the appendix) is still attached to the body and is moving
appropriately with other body parts
The Living Human Brain: The
Neurosurgeons
Insights from Wilder Penfield
(Neural Cartographer)
Construct MAP of brain areas
Localization of various functions
that displayed:
Muscular contractions
Tingling sensations in selected body parts
Mental phenomena – Hearing Music
Memory - Recall of past experience
Psychical states – feelings of fear, loneliness
Neuroscience:
Modern Technology &
Research
Evidence from Brain Mapping Research
: Insights into Human Cognition
Disorders
4 more items...
Language
2 more items...
Concept
Categories
1 more item...
Memory
2 more items...
Brain Mapping Techniques
Functional
2 more items...
Structural
2 more items...
Brain Mapping
Attempt to provide a
complete picture about
how the brain works
2 more items...
Continued brain exploration based on opportunities
presented while treating patients (i.e. epileptic seizures,
brain tumors)
Brain Research from Neurosurgeons
Discovered that stimulations could also trigger
sensation, without muscular movements
Sought to investigate ‘the pulsation of the brain’
Procedure conducted while the patient was still
conscious
inserted the electrode (with weak electric current)
in the human brain
Findings = stimulating localized brain areas could
produce specific muscular contractions
Conscious patients could provide information during
procedure
Insights from Wilder Penfield
Surgeon + experimenter
Professor of neurology
400 operations on the brain.
Patients with epileptic seizures and brain tumors.
Insights from Donald O. Hebb
Attempted to explain how neural circuits work
(i.e. underpins cognitive processes)
Any two cells or systems of cells that are repeatedly active at the
same time will tend to become ‘associated’, so that one activity
facilitates the other
Cell Assembly
Brain Research: 1960s & Beyon
d
The visual system
David Hubel
Torsten Wiesel
Connections between the eye and cortical neurons are
permanently disrupted when kitten is deprived of the
use of one eye for a period, by closing its eyelid.
That particular brain area becomes impeded.
Visual field = an area that is visible to you while
your eyes are not moving
Area of the visual field that will produce a response
in a given cell = receptive field
Hubel & Wiesel found neurons that specially detect
certain features => feature detectors
E.g.: optic nerves of frogs have “bug detectors.”
Connections to neuron impeded when the use of
the organ (i.e. eye) is also impeded
Split-Brain Research
Roger Sperry
Research that includes severing of the corpus
callosum
Research by removing the corpus callosum was originally
done on monkeys.
After surgery, the monkeys behaved normally.
Procedure was then employed with human patients who
suffered from epileptic (sawan) seizures
Investigated the role of the two hemispheres
In a normal brain, stimuli entering one hemisphere
is rapidly communicated by way of the corpus
callosum to the other hemisphere
Brain functions as a unit
Specialization of the hemispheres
Right side
Recognition of faces
and patterns
Art
Rhythm
Visual
Creativity
Synthesis
Left side
Reasoning
Language
Writing
Reading
Logic
Mathematics
Linear
Analysis
For his split-brain research, Roger Sperry shared
the 1981 Noble Prize in Physiology and Medicine
with David Hubel and Torstein Wiesel.
if the two hemispheres of the brain are
separated by severing the corpus callosum
the transfer of information between the
hemispheres ceases.
the coexistence in the same individual of two
functionally different brains can be
demonstrated.
https://coggle.it/diagram/XbSj-afCTQd7nSVk/t/neuroscience-research-contribution-to-cs
Neurology
Medicine
Artificial Intelligence
:bar_chart: :robot_face: :chart_with_downwards_trend:
Machine Learning Models
Machine Learning - Approaches
Deep-learning Algorithms
Concerned with building larger and complex neural networks, with many of them concerned with semi-supervised learning problems. These are updated form of Neural Networks
Deep Boltzmann Machine - DBM
Convolutional Neural Network - CNN
Deep Belief Networks - DBN
Stacked Auto-Encoders
NN & DL papers
Bayesian Algorithm
Methods that apply Bayesian Theorem for problems such as classification and regression
Bayesian Belief Network (BBN)
Gaussian Naive Bayes
Naive Bayes
Bayesian Network (BN)
Multinomial Naive Bayes
Averaged One-Dependence Estimators(ADDE)
Decision Trees
Decision tree methods construct a model of decisions made based on actual values of attributes in the data. Decision trees are trained on data for classification and regression problems.
Iterative Dichotomiser 3 (ID3)
M5
Classification and Regression Tree (CART)
Decision Stump
Conditional Decision Trees
Chi-squared Automatic Interaction Detection (CHAID)
Ensembles
Methods that learn from weaker models
Gradient Boosting Machines - GBM
Boosting
Bagging - Bootstrapped Aggregation
AdaBoost
Random Forest
Gradient Boosting Regression Trees - GBRT
Instance-based Algorithms
Such methods typically build up a database of example data and compare new data to the database using a similarity measure in order to find the best match and make a prediction. For this reason, instance-based methods are also called winner-take-all methods and memory-based learning.
kNN - k Nearest Neighbours
LVQ - Learning Vector Quantization
SQM - Self Organizing Map
LWL - Locally Weighted Learning
Artificial Neural Network
These are inspired by the structure and function of human brains. Such algorithms, determine the importance / weights for each variable to determine the output for a given incident
Types / Cost Function
Radial Basis Function Network (RBFN)
Back Propagation
Hopfield Network
Perceptron
NN & DL papers
Neural Net Arch Genealogy
CNN
Semantic Segmentation
4 more items...
Super-resolution
7 more items...
Object Detection
7 more items...
AlexNet, '12.12
DenseNet, '16.08
TTS
1 more item...
SENet: Squeeze-and-Excitation Networks, '17.09
ResNet, '15.12
GoogLeNet, '14.09
VggNet, '14.09
Generative Models
Latent variable models
2 more items...
Autoregressive models
5 more items...
Memory Networks
Neural Programming
7 more items...
End-to-End Memory Network,'15.03
Memory Networks,'14.10
DMN: Dynamic Memory Network, '16.03
,
DMN+, '16.04
Reinforcement Learning Algorithms
A3C, '16.02.06
c51, '17.10.27
DARLA, '17.07.26
ACTKR, '17.08.17
Capsule Net, '17.10
RNN
LSTM, '97.11
S2S: RNN Encoder-Decoder, '14.06
1 more item...
ACT: Adaptive Computation Time, '17.05
GRU, 14.11
https://coggle.it/diagram/Wf5mYoJbsgABUF9P/t/neural-net-arch-genealogy
Regression Algorithms
In such algorithms, the relationship of the variables (Dependent and Independent) is iteratively refined using a measure of error
Ordinary Least Squares Regression
Multivariate Adaptive Regression Splines (MARS)
Linear Regression
Logistic Regression
Stepwise Regression
Locally Estimated Scatter plot Smoothening (LOESS)
Regularization Algorithms
These are similar to regression methods that penalizes models based on their complexity, favoring simpler models that are also better at generalizing
Least Angle Regression (LARS)
Ridge Regresion
Least Absolute Shrinkage and Selection Operator (LASSO)
Elastic Net
Association Rule Learning Algorithms
Such algorithms generate rules based on historical records, for eg: Whenever someone buys bread in a store, quite often people buy milk along with it. Such association rule are generated using such algorithms
Eclat algorithm
Apriori algorithm
Clustering Algorithms
Grouping values together based on their distance with each other in a multi dimension space
k-Medians
Partition Around Medioids (PAM)
K-Means
Hierarchical Clustering
https://coggle.it/diagram/WlsLPFN30QABoeQB/t/machine-learning-approaches
Rule System
Repeated Incremental Pruning to Produce Error Reduction(RIPPER)
Cubist
One Rule(OneR)
Zero Rule(ZeroR)
Dimensionality Reduction
Principal Component Regression(PCR)
Projection Pursuit
Principal Component Analysis(PCA)
Linear Discriminant Analysis(LDA)
Partial Least Squares Regression(PLSR)
Quadratic Discriminant Analysis(QDA)
Partial Least Squares Discriminant Analysis
Mixture Discriminant Analysis(MDA)
Sammon Mapping
Flexible Discriminant Analysis(FDA)
Regularized Discriminant Analysis(RDA)
Multidimensional Scaling(MDS)
Machine Learning - Approaches
(presented another simpler way - 2019)
Classical Learning
Supervised
Regression
Linear
Polynomial
Ridge / Lasso
Classification
kNN
Naive Bayes
SVM
Decision trees
Logistic Regression
Unsupervised
Clustering
Fuzzy C-Means
Mean Shift
k Means
DBSCAN
Agglomerative
Pattern Search
Euclat
Apriori
FP-Growth
Dimention Reduction (generalisation)
t-SNE
PCA
LSA
SVD
LDA
Neural Networks & Deep Learning
Perceptrons (MLP)
Autoencoders
seq2seq
Generative Adversial Networks (GAN)
Convolutional Neural Networks
DCNN
Recurent Neural Networks (RNN)
LSM
LSTM
GRU
Reinforcement
Genetic Algorithms
A3C
SARSA
Q-Learning
Deep Q-Network (DQN)
Ensemble Methods
Stacking
Bagging
Random Forest
Boosting
Adaboost
Catboost
LightGBM
XGBoost
Maths
Math for Machine Learning
Matrices
:bookmark_tabs:
Matrix product properties
Matrix Products
Distributativity
A(B+C) = AB +AC
Not commutativity
AB!=BA
Associativity
A(BC)=(AB)C
The Identity Matrix
IA=A
All ones on the diagonal
Properties of the Hadamard Product
Commutativity
AoB=BoC
Associativity
A(BC)=(AB)C
Distributativity
A(B+C) = AB +AC
Linear dependence
Definition
lies in lower dimentional space
if there are some a-s, that
a1*v1+a2*v2+...+ak*vk=0
Example
a1=1, a2=-2, a3=-1
det(A)=0 only if columns of A are linearly dependent
Dot product and how to extract angles
Hyperplane
- is the thing orthogonal to a given vector
perpendicular line in 2D
perpendicular surface in 3D
Decision plane
2D
3D
1D
Key Consequences
vw<0 vw>0
Orthogonality
v*w=0
Dot Product
- the sum of the products of the corresponding entries of the two sequences of numbers.
Angles
Matrix multiplication and examples
If A is a matrix where the rows are features wi and B is a matrix where the columns are data vectors vj then the i,j-th entry of the product is wi*vj, which is to say the i-th feature if the j-th vector.
In formulae: if C=AB, where A is an
n x m
matrix and B is a
m x k
matrix, then C is a
n x k
matrix where
Well defined:
Geometry of matrix operations
The Determinant
det(A) is the factor the area is multiplied by
det(A) is negative if it flips the plane over
Intuition from Two Dimentions
Suppose A is 2x2 matrix (mapping R^2 to itself). Any such matrix can be expressed uniquely as a
stretching
, followed by a
skewing
, followed by a
rotation
Any vector can be written as a sum scalar multiple of two specific vectors
A applied to any vector
Determinant computation
Larger Matrices
m determinants of (m-1)x(m-1) matrices
computer does it simplier Q(m^3) times
called matrix factorizations
The Two-by-two
det(A)=ad-bc
Matrix invertibility
When can you invert?
it can be done only when det != 0
How to Compute the Inverse
A^(-1)*A=I
Hadamard product
An (often less useful) method of multiplying matrices is element-wise
AoB
Vectors
:arrow_right:
Measures of Magnitude
Norm Properties
All distances are non-negative
Distances multiply with scalar multiplication
If I travel from
A
to
B
then
B
to
C
, that is at least as far as going from
A
to
C
(Triangular Inequality)
Types of Norms
Lp-Norm
Euclidean Norm (L2-Norm)
L1-Norm
L0-Norm
Despite the name, it's
not
a norm
It's a number of non-zero elements of a vector
L∞-Norm
Norms
- are measures of distance
Geometry of Norms
Geometry of Column Vectors.
Operations
Subtraction as Mapping
Scalar Multiplication
Addition as Displacement
Vectors as Directions
Probability
:game_die:
The Gaussian curve
Standard Gaussian
(Normal Distribution) Density
E[X]=0
Var[X]=1
Key Properties
Central limit theorem
is a statistical theory states that given a sufficiently large sample size from a population with a finite level of variance, the mean of all samples from the same population will be approximately equal to the mean of the population.
Maximum entropy distribution
Amongst
all
continuous RV with E[X]=0, Var[X]=1. H(X) Entropy is maximized uniquely for X~N(0,1)
Gaussian is the
most
Randon RV with fixed mean and variance
General Gaussian Density
Visualizing Probability
using Venn diagram
Inclusion/Exclusion
Relative complement
of A (left) in B (right)
Symmetric difference
of two sets
Intersection
of two sets
Absolute complement
of A in U
Union
of two sets
General Picture
Sample Space <-> Region
Outcomes <-> Points
Events <-> Subregion
Disjoint events <-> Disjoint subregions
Probability <-> Area of subregion
Chebyshev’s inequality
For
any
random variable X (no assumptions) at least 99 of the time
Terminology
Outcome
A single possibility from the experiment
Probability
Fraction of an experiment where an event occurs
P{E} є [0,1]
Event
Something you can observe with a yes/no answer
Capital E
Sample Space
The set of all possible outcomes
Capital Omega
Entropy
Examples
Two coins
Entropy
= 2 bits of randomness
T
TT (1/4)
TH (1/4)
H
HH (1/4)
HT (1/4)
A mixed case
Entropy
= 1.5 bits of randomnes =
=1/2(1 bit) + 1/2(2 bits)
T
TH (1/4)
TT (1/4)
H (1/2)
One coin
Entropy
= one bit of randomness
T (1/2)
H (1/2)
Entropy
(
H
)
Examine the Trees
if we flip n coins, then P=1/2^n
# coin flips = -log2(P)
The Only Choice was the Units
firstly you need to choose the base for the logarithm.
If the base is not 2, then
Entropy
should be divided by log2 of the base
Bayes’ rule
Posterior odds = ratio of probability of generating data * prior odds
can be leveraged to understand
competing hypotheses
odds is fraction of two probabilities
i.e. 2/1
Axioms of probability
Something always happens.
The fraction of the times an event occurs is between 0 and 1.
If two events can't happen at the same time (disjoint events), then the fraction of the time that
at least one
of them occurs is the sum of the fraction of the time either one occurs separately.
Independence
A and B are independent if P{AnB}=P{A}*P{B}
Two events are
independent
if one event doesn't influence the other
Random variables
is a function X that takes in an outcome and gives a number back
Standard Deviation
Discrete X takes only at most countable many values, usually only a finite set of values
Expected Value
mean
Variance
how close to the mean are samples
Continuous random variables
For many applications ML works with
continuous random variables
(measurement with real numbers).
Probability density function
Building machine learning models
Maximum Likelihood Estimation
Given a probability model with some vector of parameters (Theta) and observed data
D
, the best fitting model is the one that maximizes the probability
Conditional probability
If I know B occurred, the probability that A occurred is the fraction of the area of B which is occupied by A
Intuition:
The probability of an event is the expected fraction of time that the outcome would occur with repeated experiments.
Univariate Derivatives
:silhouette:
Newthon's method
Idea
Minimizing f <---> f '(x)=0
Now look for an algorithm to find the zero of some function g(x)
Apply this algorithm to f '(x)
Update Step for Minimization
To minimize
f
, we want to find where
f '(x)=0
and thus we may start with some initial guess x0 and then iterate Newton's Method on
f '
to get
Update Step for Zero Finding
we want to find where
g(x)=0
and we start with some initial guess x0 and then iterate
Relationship to Gradient Descent
A learning rate is adaptive to f(x)
Pictorially
g(x) x such that g(x)=0
Computing the Line
line: on (x 0, g(x 0))
slope g '(x 0)
y=g '(x 0) (x-x 0)+g(x 0)
solve the equation y=0
Maximum Likelihood Estimation
find
p
such that
Pp(D)
is maximized
Gradient Descent
ALGORITHM
1
. Start with a guess of X0
2.
Iterate through
- is learning rate
3.
Stop after some condition is met
if the value if x doesn't change more than 0.001
a fixed number of steps
fancier things TBD
Issues
:
Sometimes we can
circumvent
this issue.
recall that an
improperly chosen
learning rate will cause the entire optimization procedure to either
fail
or
operate too slowly
to be of practical use.
how to pick eta
As simplistic as this is,
almost all
machine learning you have heard of use
some version
of this in the
learning process
Goal
: Minimize f(x)
Derivative
Rules
Chain Rule
Alternative
Quotient Rule
Product Rule
Sum Rule
Can be presented as:
Interpretation
Most usable
http://hyperphysics.phy-astr.gsu.edu/hbase/Math/derfunc.html
let's approximate
better approximation
f(x +є) = f(x) + f'(x)є
Second Derivative
f''(x)
shows how the slope is changing
max
-> f '' < 0
min
-> f '' > 0
Can't tell
-> f '' = 0
proceed with higher derivative
Multivariate Derivatives
:silhouettes:
Convexity
the function is convex if the line between two points stays above
Derivative Condition
Hf is called positive semi-definite if
And this implies f is convex
A Warning
When going to more complex models i.e. Neural Networks, there are many local minima & many saddle points.
So they are not convex
Benefits
When a function is convex then there is
a single
unique
local minimum
no maxima
no saddle points
Gradient descent
is guarantied to find the global minimum with
small enough
Newton's Method always works
Opposite is Concave
Matrix Calculus
The Gradient
Key Properties
points in the direction of
maximum increase
-
points in the direction of
maximum decrease
at local max & min
Definitions
The Gradient
collection of partial derivatives
Vector derivative
Matrix derivative
Visualize
Gradient Descent
Level set
Newton Method
The computational complexity of inverting an
n x n
matrix is not actually known, but the best-known algorithm is
O(n^2.373)
For high dimensional data sets, anything past linear time in the dimensions is often impractical, so Newton's Method is reserved for a few hundred dimentions at most.
Second Derivative
Hessian
2D intuition
Critical points
=0
det(Hf) > 0
further investigation
3 more items...
det(Hf) < 0
saddle point
det(Hf) = 0
unclear
need more info
Hf=
Hf=
Hf=
If the matrix is diagonal, a positive entry is a direction where it curves up, and a negative entry is a direction where it curves down
For
, there is
many derivatives
Trace
sum of diagonal terms tr(Hf)
Partial Derivatives
is a measure of the rate of change of the function... when one of the variables is subjected to a small change but the others are kept constant.
Example
:
If you have a function
where n - n-dimensional vector,
Then it's a function of many variables.
You need to know how the function responds to changes in all of them.
The majority of this will be just bookkeeping, but will be terribly messy bookkeeping.
https://coggle.it/diagram/XbOer64hAVFRviPP/t/math-for-machine-learning
others
Graph Theory
:deciduous_tree:
↑ Maths / Linguistics ↓
1 more item...
Gödel's incompleteness theorems
:smiling_imp:
Dynamical Systems
:tornado:
other maths
Data
Databases
Big Data
Workload management
Slurm
Commands
Job Management
squeue
sinfo
scontrol
scancel
Accounting
Job Submission
salloc
sbatch
srun
Daemons
Environement Variables
https://slurm.schedmd.com/pdfs/summary.pdf
GPU / CPU
Google Collab
Open Data
Datasets
MNIST
Data Mining
Clustering
Expectation Maximization
k-Medians
Hierarchical Clustering
Grid-Clustering
k-Means
DBSCAN
Agent mining
Text mining
Anomaly/outlier/change detection
Genetic algorithms
Structured data analysis
Intention mining
Big data
Association rule learning
Sequence mining
Time series analysis
Informatics
Engineering / Libraries for ML
numpy
tensorflow
panda
matplotlib
keras
pytorch
gensim
nltk
Web programming (not related to cogsci but you never know)
Ethics
Applications
Bioinformatics
Linguistics (NLP)
NLP
Natural language Processing
Open Source Libraries
Apache OpenNLP
Mallet
Natural Language Toolkit
Standford NLP
Terminology
Morphology
Phonology
Pragmatic
Discourage
Semantic
Morpheme
Syntax
World Knowledge
History
1970s
Conceptual Ontologies
1950s
Turing Test and George Town Experiment
1980s
Machine learning algorithm
1960s
Elizabeth pyschotherapist
Areas
Named entity recognition
Part of speech
Optical charaacter recognition
Applications
Extracting data from set
MachineTranslation
Database Access
Information Retrieval
Text Categorization
Components
Natural Language Understanding
deals with machine reading comprehension
Natural Language Generation
a translator that converts data into a natural language representation
Introduction
The field that covers computer understanding and manipulation of human language, and it’s ripe with possibilities for newsgathering
The way for computers to analyze, understand, and derive meaning from human language in a smart and useful way
The field of study that focuses on the interactions between human language and computers
Steps
Syntactic Analysis concerns the proper ordering of words and its affect on meaning
Pragmatic Analysis concerns the overall communicative and social context and its effect on interpretation
Lexical analysis involves dividing a text into paragraphs, words and the sentences
Semantic Analysis concerns the (literal) meaning of words, phrases, and sentences
Discourage Integration concerns the sense of context
https://coggle.it/diagram/XbQngkUb1ndwn-sJ/t/natural-language-processing
Natural language processing
Syntax
Terminology extraction
Lemmatization
Stemming
Parsing
Part-of-speech tagging
Word segmentation
Morphological segmentation
Sentence breaking
Semantics
Lexical semantics
Machine translation
Topic segmentation and recognition
Question answering
Natural language understanding
Word sense disambiguation
Optical character recognition (OCR)
Named entity recognition (NER)
Natural language generation
Recognizing Textual entailment
Sentiment analysis
Relationship extraction
Discourse
Coreference resolution
Discourse analysis
Automatic summarization
Speech
Speech segmentation
Speech recognition
Text-to-speech
Chatbots
Robotics
Embedded Systems
Logic gates
Microcontrollers
RTOS
Reinforcement Learning
Computer Vision
Robotics
Developmental robotics
Human-robot interaction
Robotic Voice
Artificial Emotions
Facial Expression
Social Intelligence
Gestures
Personality
Speech Recognition
Evolutionary robotics
Behavior-based robotics
Cybernetics
Cognitive
Educational
Swarm
Soft
Computer Vision
Scene Reconstruction
System methods
Image-understanding Systems
Image Restoration
Recognition
Optical character recognition (OCR)
Optical character recognition
Pose estimation
Content-based image retrieval
Handwriting recognition
2D Code reading
Shape Recognition Technology (SRT)
Facial recognition
Haar features (Viola & Jones)
Motion Analysis
Tracking
Egomotion
Optical flow
Image Generation
GANs
ImageNet Large Scale Visual Recognition Challenge Results shows that deep learning is surpassing human levels of accuracy.
https://deepindex.org/
Game Bot
Autonomous Vehicle
Computational Neurosciences
Cybernetics
Representations
Knowledge Reasoning / Knowledge Representation
Planning
Probabilistic Planning
Domain Independent Planning
Preference-based Planning
Classical Planning
Temporal Planning
Planning Domain Modelling Languages
Ontologies
Automated Reasoning Engines
Theorem Provers
Inference Engines
Expert System
Semantic Nets
Frames Rules
System Architecture
Communities
Papers
People
you're on the right way, go on
yup, keep going, it is just kinda far
Google Brain
Computer System for Machine Learning
Tensorflow
TPU
TPU 2.0
TPU 3.0
Rajat Monga
Principal Scientist
Tensorflow/OpenML library
Paul Barhm
Principal Scientist
TPU/Low level Tensorflow
Patrick Nguyen
Senior Staff Research Scientist
17 more items...
Natural Language Understanding/Perception
Sequence to Sequence Learning
with Neural Networks (2014)
A Neural Network for Machine Translation, at Production Scale
Zero-Shot Translation with Google’s Multilingual Neural Machine Translation System
Tacotron 2’s model architecture
Yonghui Wu
:check:
Principal Scientist
-Information Retrieval; Ranking; ML
-NLP
-Machine translation
Samy Bengio
Principal Scientist
-Statistical ML
13 more items...
Machine Learning Algorithm and Techiques
capsules
sparsely-gated mixtures of experts
hypernetworks
new kinds of multi-modal models
-symbolic and non-symbolic learned optimization methods
back-propagate through discrete variables
Geoffrey Hinton
Capsule Neural Network
Co- founder/ principical scientist
Algorithm and theory-
-Machine intellengce
-NLP
-Speech processing
13 more items...
Healthcare and Bioscience
Ophthalmology and digital pathology
Diagnosing Diabetic Eye Disease
Assisting Pathologists in Detecting Cancer
Genomics: DeepVariant Highly Accurate Genomes with DNN.
Greg Corrado
Senior research scientist/Co Founder of Google Brain
Biological Neuroscience
-Artificial Intelligence
-Scalable ML
24 more items...
Robotics
Deep Reinforcement Learning for Vision-Based Robotic Grasping
Time-Contrastive Networks:
Self-Supervised Learning from Multi-View Observation
Vincent Vanhoucke
Principal Scientist
-Machine Intellegence
-Machine Precption
-Robotics
32 more items...
PAIR: Big Picture Visualization Group
Facets/ Facets Dive
Distill
Quick, Draw!
Fernanda Viegas:
Senior Research Scientist- big picture group- data visualization
-Distributed systems and parallel computing
-Education innovation
-Human- Computer Interaction and Visualization
-Machine Intelligence
Martin Wattenberg
Research Scientist/Co-founder of big picture group
10 more items...
Music and Art Generation
Megenta project
Developing new deep learning and reinforcement learning algorithms for generating songs and other art materials
Douglas Eck
Principal Scientist
-human computer interaction and visualization
-Machine intelligence
-Machine Perception
-NLP
8 more items...
https://coggle.it/diagram/XbTHY-fHayl_rmbP/t/google-brain
Google Deepmind
Facebook FAIR
Microsoft
Open AI
Baidu
Nvidia
Intel Nervana
IBM
Apple
Amazon
Uber
Salesforces Metamind (?)
ML Optimizers
gradient descent
SGD
(calculated in every sample of the ds per epoch)
MBGD
(same than SGD but with mini batchs)
NAG
(reduce error by using futur steps)
Momentum
(throwing ball downhill, go faster if in right direction)
Adaptive learning rate
Adagrad
(adapts learning rate to the parameters based on previous gradients, but has learning rate shrinking pb)
RMSProp
(decaying average of all past square gradients)
Adam
(RMSProp + bias correction + momentum)
Nadam
(Adam + NAG)
Adadelta
(decaying average of all past square gradients)
BGD
(calculated on entire ds)
Deontological approach (~else/if)
Human Computer Interaction
:silhouette: :left_right_arrow: :computer:
Engineering
Type of Interactions
Haptics
Lederman & Klatzky
channels
gestural channel
semiotic
epistemic
ergotic
proprioception
touch
kinestesia
Visuals
UX Design
Design of HCI
task analysis
HTA
stopping rules
hierachis of goals and subgoals
tasks
representation of analysis
Goals
GOMS
goals
methods
selection rules
operations
levels of analysis
micro
cognition
intermediate
task
macro
environment
interface surveys
labelling surveys
enviromental surveys
walkthrough approaches
cognitive walkthrough
data collection
verbal protocols
questionaries
structured interviews
Ergonomic checklists
user analysis
user population
interesting characterisics
age
education
IT knowledge
sex
physical limitations
personae
not real person
very detailed
requirements gathering
indirect observation
direct obsveration
stakeholders
e.g. managers
interface specification
includes
layout
state changes
usage scenarios
error messages
behaviour
description of interaction
description of physical interface
Specification Vs Prototyping
prototyping
represent
system
task
user
types
paper
low fidelity
interaction not real
easy to set up
full
rarely used
lots of time coding
horizontal
longer to build
give users scope for making mistakes
slide show
limited representation
good simulation
e.g. visual basic
high fidelity
vertical
one part of system fully implemented
HCI software process
https://coggle.it/diagram/XbOiWnoA8nxvssP3/t/hci-revision
GUI
Graphical User Interface
WIMP
Windows Icon Menu Pointing
Fitz Law
Modes
other types of interactions
"beating Fitz Law"
Space scale diagrams
bubble cursor
zoomable UI
multiscale pointing
Goal passing
Steering Law
XR
Virtual Reality
Applications of VR
Data visualization, science, archaeology
Industrial design
Entertainment and art
Health and wellbeing
Psychological research and mental health therapy
Education and training
The MIND project
Used at the clinic, as part of a regular 'face-to-face' consultation
Experience the mind's dynamics, learn powerful therapy tools and manipulate mental processes with your own hands
Modular design allows to build a suite of apps over time
History
Started since 1950s: Sensorama
Current VR apps: games, simulation
Gartner Hype cycle for emerging technologies: innovation trigger -> peak of inflated expectations -> trough of disillusionment -> slope of enlightenment -> plateau of productivity
Current day: mobile VR headsets, wired VR headsets
Introduction
3-D audio
Inputs and outputs: computer animation is the same as desktop games, but the camera position is controlled by the player (translation and rotation of camera is affected by the player's head movements)
An embodied sensory experience of a synthetic environment
Hand controller: to move objects in the virtual space
Head mounted display: Need to track player's head movements to change position of camera in the virtual space
Conditions addressed by VR in mental health
Anxiety (bulk)
Psychosis
Depression (almost nothing)
Enabling exposure therapy, Thought & behaviour therapy, to be more widely used
https://coggle.it/diagram/XbOoBHoA8pP5ssc9/t/18-virtual-reality
Augmented Reality
IOT
BCI
Gestural
Multimodal
Conceptual Models
Affordances / Signifiers
(JJ Gibson/ Don Norman)
Metaphors
3 rules (Don Norman)
visibility
mapping
feedback
3 dimentions of an interface
Reuse (interaction/visualisation)
Reification (objects)
Polymorphysm (commands)
Speech
McGurk effect
electrical engineering
mechanical engineering
Ressources
The Encyclopedia of Human-Computer Interaction, 2nd Ed.
15.2 From Usability to User Experience - Tensions and Methods
15.2.3 The Long and Winding Road: Usability's Journey from Then to Now
15.2.1 New Methods, Damaged Merchandise and a Chilling Fact
15.2.4 Usability Futures: From Understanding Tensions to Resolving Them
15.2.2 We Can Work it Out: Putting Evaluation Methods in their (Work) Place
15.1 From First World Oppression to Third World Empowerment
Usability Evaluation
the extent to which an interactive system is easy and pleasant to use
evaluation
designing requires making choices.
Evaluations have to be designed
choose how to evaluate.
propositions
evaluation methods
measurable property
Evaluation methods and metrics are thoroughly documented in the Human-Computer Interaction research and practitioner literature.
15.1.1 The Origins of HCI and Usability
time
1960 -> minicomputers
1980 -> no / basic training on operating systems and application
1946 -> born of computer -> specialists of expensive centralised equipment
problem
HCI was born
software design practice -> assume knowledeable and competent users
15.1.2 From Usability to User Experience via Quality in Use
user experience
usability
how inherently usable an interactive system was -> how well it fitted its context of use.
Quality in use
15.1.4 From HCI's sole concern to an enduring important factor in user experience
15.1.3 From Trouble with Computers to Trouble from Digital Technologies
1999 -> hole in the wall -> children learn computers
15.3 Locating Usability within Software: Guidelines, Heuristics, Patterns and ISO 9126
15.3.2 Manageable Guidance: Design Heuristics for Usable User Interfaces
15.3.3 Invincible Intrinsics: Patterns and Standards Keep Usability Essential
15.3.1 Guidelines for Usable User Interfaces
15.4 Locating Usability within Interaction: Contexts of Use and ISO Standards
15.4.1 Contextual Coverage Brings Complex Design Agendas
15.5 The Development of Usability Evaluation: Testing, Modelling and Inspection
https://coggle.it/diagram/XbOhdK4hAcR1viaq/t/the-encyclopedia-of-human-computer-interaction%2C-2nd-ed
Ch 2 - Understanding and conceptualising interaction
Paradigms, visions, theories, models, frameworks
Frameworks
Norman's framework
User
How the user thinks the system works
Designer
How the designer thinks the system should work
System
The system image is how the interface, manuals, etc communicate the workings of the system to the user
Numerous frameworks exist
Variety of forms - steps, questions, concepts, challenges, principles, tactics, dimensions
Offer advice to designers - what to design or look for
Traditionally based on theories of human behaviour, increasingly derived from actual design experience and user studies findings
Theories
Cognitive
Social
Organisational
Different theories have entered the HCI realm over the last 30 years, as a way to analyse and predict the performance of users of computer base interfaces
Paradigms
How findings should be analysed and interpreted
The phenomena to be observed
The questions to be asked and how they should be framed
Augmentation (late 1990s) - sensors + algorithms in/on homes, public buildings, bodies - parking lots which tell you where the open bays are, how well you slept and whether your blood pressure is too high / Internet of Things
Ubiquitous tech (Weiser 1991) - tech embedded in everyday objects, in an unobtrusive way
1980s - user centred applications for desktops / screens; WIMP = Windows, Icons, Menus, Pointer
Visions
Visions of the future drive research and design
The inventor of the mobile phone was inspired by Star Trek, tablet like devices were seen in
2001: A Space Odyssey
(made in 1968)
Models
They are usually derived from a theory from a contributing field such as psychology
Norman's models of interaction design (e.g. 7 stages of action model, model of emotional design) were based on cognitive science
Keystroke model (Card, Moran and Newell) was also based on cognitive science
Conceptual models
Working strategy and framework of general concepts and their interrelations
Concepts that people are exposed to through the product including task-domain objects
Attributes e.g. name, description, date added
Operations that can be performed on them, e.g. add, update, delete, move, organise
Mappings between concepts and user experience e.g. having easy access to useful sites, being able to group them in your own categories so that they are easy to find, being able rename them so that they are easy to recognise
Relationships e.g. categories, groups, tags
Metaphors and analogies that enable people to understand what the product is and how to use (e.g. bookmarking)
Johnson and Henderson's definition
"Enables designers to straighten out their thinking before they start laying out their widgets"
"High level description of how a system is organised and operates"
Examples of conceptual models and interface metaphors
Operating systems use the physical office idea of "desktops", "folders", "files", "trashcans" metaphors
Task management tools use "post-its" metaphors
Online shopping sites use the physical shopping mall concept with "stores", "shopping carts", "checkout" metaphors
E-readers use "bookshelves" metaphors
Google's Material Design uses the paper metaphor
Problem space
Team effort
Operational impact
Financial implications / budget
Technical solutions / impact
Customer impact
Overview
Understand how this can be improved
Understand the current user experience
Is there a problem to be solved?
(Don't just start designing)
Research first
Framework
Could your design create new problems? (not from the textbook)
How does the proposed design solve these problems?
What is it?
Why do you believe there is a problem?
If there is no problem but you are trying to improve the UX, how do you believe your design achieves this?
Is there a problem?
Benefits
Open mindedness - prevent the design from becoming narrowly focused too early
Common ground - establish a common language to reduce confusion and misunderstandings
Orientation - ask specific questions about how the conceptual will be understood by users
Assumptions and claims
Claim: We only need to support smartphones. Assumption: All of our customers surely have smartphones by now. Reality: 20% of our customers still use feature phones.
Claim: We need to redesign our search tool. Assumption: Our search tool is really unusable and that's why nobody uses it.
Reality: The type of content on our site is not well suited to search.
Interaction types
Exploring
Moving through a physical environment (with sensors), e.g. Apartheid Museum, or a #D virtual environment e.g. Minecraft, Subway Surfer
Manipulating
Core principles for manipulation
Continuous representation of objects and actions of interest
Physical actions / pressing buttons instead of issung complex commands
Rapid reversible incremental actionawith immediate feedback
Moving, opening/closing, zooming, dragging items using a physical device (.e.g mouse, Wii) or gestures, or fingers against a touch screen
Conversing
Spoken or typed dialogue, e.g. search using voice input or Web browsers for the speech impaired
Instructing
E.g. print a file (PC), give me a coldrink (vending machine), delete a photo (phone)
MICE
Interface metaphors
Provide familiar entities that enable people to understand the underlying conceptual model, but do not usually function in exactly the same way, which can confuse people
Shopping cart - I put items I want to buy in there, but in the real world I have to buy them right away or the cart is emptied; if I get books from the library I don't put them into a cart
Trashcan - for getting rid of unwanted stuff - but why is it on the desktop?
Folders - I can put related items into a folder, but in the real world there is a limit to how many items
Natural Sciences
psychology
anthropology
physiology
Design
industrial design
CAD
lazer cutting
additive manufacturing
material engineering
typography
architechture
Education
:mortar_board: :bulb: :baby:
à organiser
Piaget
Bayesian Learning (Tenenbaum, Salakhutdinov)
Flow (Chiksenmihay, Nakamura)
Social constructivism / Lev Vygotsky
ZPD "zone of proximal development" ( Lev Vygotsky)
Psychology
Artificial Intelligence
Neurosciences
Human Computer Interactions
Anthropology
Linguistics
Philosophy
Education
Author:
https://twitter.com/tllhglld
State: Draft (not finished yet)
Licence: CC
To remember: One field is strong but what the other fields bring to it makes it stronger / you can do "ctrl+F" to find something
