Please enable JavaScript.

Coggle requires JavaScript to display documents.

Generative AI - Coggle Diagram

- - - - basic vectorization. Ad sempio one-hot encoding crea matrici 1-1 con il numero di elementi dei contenuti.
        
        semplici
        
        da luogo a matrici di grandi dimensioni e sparse (molti 0 e pochi 1)
      - Word embedding is a technique to encode text into meaningful vectors. The technique lets you represent text with low-dimensional, dense vectors. You don’t need the vector size as big as 20,000 like in one-hot encoding. Instead, you have between single-digit dimensions (for small datasets) and four-digit dimensions (for large datasets). Each dimension is supposed to capture a feature of a word. A higher dimensional embedding captures detailed relationships
        Considerazioni:
        
        dimensioni ridotte
        
        vettori densi
        
        training oneroso
        
        Ci sono reti neurali già pronte per creare embedding da testo
  - - - How can you measure the distance between vectors?
        Ci sono 4 diverse algoritmi per confrontare distanze tra vettori
        
        distanza di manhattan
        
        distanza euclidea
        
        coseno (si considera angolo tra due vettori)
        
        inner product (si usano proiezioni dei vettori lungo una direzione)
      - How to efficiently find the similar vectors in the vast vector space?
        Two search algorithms have seen extensive use: Brute force, which is based on an accurate exhaustive search approach, and TreeAh, which relies on an approximate tree search algorithm.
        
        The brute force algorithm typically consists of three steps: Step one: Calculate the distances from the query to the other vectors in the vector space.
        Step two: Sort all distances.
        Steph three: Find the top K nearest vectors.
        Because the size of the database can easily be in the millions or even billions, exhaustive search is impractical and the brute force algorithm becomes a computational bottleneck.
        Consider the brute force complexity, which is O(N*d), where N is the number of vectors and d is the number of dimensions.
        
        The TreeAh algorithm accelerates search by using an approximate search technique called ANN, or approximate nearest neighbor.
        ANN enables fast and scalable search with billions of embeddings by dividing the search space into multiple spaces,
        indexing the spaces using a tree structure, and trading some accuracy for a significant speedup over brute force search.
        To enhance vector search performance, you need to apply a combination of critical techniques: Reduce the search space, also known as space pruning, and Compress the vector size using data quantization.
        The second technique is data quantization.
        Data quantization is used to compress data points to save space and reduce indexing time.
        The third technique is to incorporate business logic to retrieve only the data that is relevant.
        ScaNN, Scalable Approximate Nearest Neighbor, is a combination of these techniques.
    - - This enables multimodal search applications, where users can search for information using multiple types of data.
      - Personalization and recommendation: Vector search can be used to personalize search results and recommendations by leveraging its context understanding capability
      - This is useful for natural language queries, where users may not be using precise or technical language.
- - - - Vertex AI Search
        is useful when you can scale a wide range of siloed data to retrieve accurate and trusted information, Use Vertex AI Search where you can build a Google-quality search engine on your own data and embed a search bar in your web page or application..
      - Vertex AI Conversation
        is wonderful when you can have human-like interactions to solve problems and find information
  - - - Come funzionano? In pratica il next token è generato in base a una distribuzione di probabilità.
        Se prendo token con probabilità più alta si parla di Greedy decoding.
        In Greedy decoding is a reasonable and fairly intuitive strategy, but it does have some drawbacks, such as outputs with repetitive loops of text.
        In alternativa si può scegliere di restituire un token casuale tra i più papabili.
      - Come migliorare
        
        Tuning
        Fine Tuning: retraining the model (molto costoso)
        Prompt Tuning: With Prompt tuning you can tune a vector or “soft prompt”, that gets sent prepended to your input text, every time you want to call the model.
        Prompt Design: With Prompt Design, you write descriptions of the task you want executed, and this requires no updates to the base model or weights.
        Parameter-efficient tuning: meno costosa di allenare tutto il modello, si estende LLM aggiungendo parametri. Servono molti dati/esempi.
        
        Supervised Tuning
        Supervised tuning improves the accuracy of a model by teaching it to format its output more specifically: It uses hundreds of examples to teach the model to mimic a desired output pattern.
        Supervised tuning enables the model to learn additional parameters that help it encode the necessary information
        to perform a desired task, or learn a desired behavior, which are later used during inference.
        When tuning is complete, a new model with layers that sit on top of the original model is generated.
        
        Reinforcement learning with human feedback or rhf
        rlaf optimizes language models using human specified preferences this process improves model alignment with human preferences and reduces
        undesired outcomes in tasks with complex human intuitions for example rhf helps with ambiguous tasks like writing creative
        content it involves presenting two options to a human and letting them choose their preferred one a data
        
        Model distillation
        Distilling step-by-step is a new method that allows us to train smaller task-specific models, with much
        less training data than required by standard fine-tuning or distillation approaches, that outperform few-shot prompted LLMs’ performance.
        Servono tra i 1k e 10k esempi unlabelled o centinaia di esempi labelled
        
        Prompt Design
        N-shot prompting and it comes in different variants. For example, zero-shot prompting is when you simply send an instruction without any examples, and few-shot prompting is when you send an instruction along with examples.
        Chain of Thought which consists of providing a prompt and breaking up the actions, into intermediary reasoning steps, so that the LLM can find the most correct, well thought out answer. This technique excelling at various arithmetic, commonsense, and symbolic reasoning tasks.
        ReAct and it’s a loop thinking process involving answering questions and looking up data in external systems. After a prompt, the LLM responds with a thought and an action, which is used to generate an observation.
        The thoughts are the reasoning about how to act.
        The actions are used to formulate calls to an external system, which can be Cloud Functions or external APIs.
        And the observations are the response from the external system.
        Through these interleaved thoughts, actions, and observations the LLM eventually arrives at an answer.
        
        Prompt Tips
        Label: una label è una sentenza seguita da ":".
        Prompt principalmente sono di 3 tipi: classification summarization and extraction. classification prompts make the model assign a class or a category to the text you can have the model automatically assign a category or you can provide it with a list of categories to choose from summarization prompts extract a summary of the most important information from the text you can give the model additional direction or let it choose an appropriate summary you can influence the length of its response by increasing or decreasing the maximum number of tokens when you submit your prompt and extraction
        prompts allow you to ask the model questions about a given text it will parse the text and use it to answer your questions
        
        Classifcation Prompt
        The model will select one classification for the input text rather than outputting a list of probabilities furthermore it means that if you give the model the same input text and parameters it should always give you back the same response. È possibile istruire anche un'opzione di fallback.
        
        set temperature to 0
        
        set the top K responses to one this forces the model to always pick the classification label
        
        Summarization Prompt
        Scrivi prima le istruzioni per il summary e poi a seguire testo da fare summary
        Increase the creativity of the model's responses by increasing the temperature top k and top P values and when you write your prompt tell the model the purpose
        
        Extraction Prompt
        in the text event extraction is used for extracting information about events from text such as project milestones and product launch dates and question answering this
        allows you to ask the model questions about text that you provide giving you contextual responses with citations from the text this can save you a
        
        set temperature to 0
        
        set the top K responses to one this forces the model to always
        
        Chat Prompt
        analyze data for you there are three principal components of a chat prompt that you should take into consideration
        messages context and examples messages contains a list of author content pairs that distinguish whether the message content originated
        from the user or the model the model will respond to the last message in the list using all other messages in the list as context for its response thereby making the user's chat experience more conversational
        Components:
        
        message
        
        context (history)
        
        examples
        
        Best practices:
        
        start by giving the chatbot an identity and a Persona this ells the model to role play
        
        give rules for the chatbot to follow so that you can limit Its Behavior
        
        add rules that prevent the exposure of context information
        
        test your chatbot and add rules to counteract undesirable behaviors
        
        add a rule to reduce hallucinations and to help the chatbot give more factual answers
        
        Decoding Strategies
        There are several methods for picking the output token, and these methods are called decoding strategies.
        Key parameters: Temperature, Top K and Top P. All three parameters are available in the Vertex AI console, and also in the API.
        Temperature is a number that can be used to tune the degree of randomness. Lower temperatures mean less randomness, and higher temperatures mean more randomness.
        Top K, where you sample from a shortlist of the top K tokens. This approach allows the other high-scoring tokens a chance of being picked. It’s important to note that setting the top-k parameter to 1, will result in greedy decoding.
        With top P, you choose from the smallest possible set of words, whose cumulative probability is greater than, or equal to P. P acts after K, meaning Top P will dynamically set the size of the shortlist tokens, established
        
        Evaluation
        Come valutare la bontà di un LLM. Task complicato
        perché dipende dall'attività
        
        Classification
        Si cerca una risposta deterministica, caso più facile.
        Per valutare LLM si possono usare:
        
        Accuracy
        
        Precision
        
        Recall
        
        F1 score
        
        Macro F1 o Micro F1
        
        Text Generation
        Task complicato perché risposta è valutata soggettivamente.
        Es. un riassunto.
        Possibili metriche:
        
        ROUGE-L
        
        BLEU score
        
        Question answering
        Via di mezzo, perché deterministica ma dipende com'è
        costruita la risposta.
        Metrica: exact match