Please enable JavaScript.

Coggle requires JavaScript to display documents.

generative AI - Coggle Diagram

- - - - models repository
        
        https://huggingface.co/
        
        https://civitai.com/
      - UI
        
        Web server UIs
        
        AUTOMATIC1111
        
        https://github.com/AUTOMATIC1111/stable-diffusion-webui
        
        Extensions
        
        https://stable-diffusion-art.com/automatic1111-extensions/
        
        from the article
        
        12 more items...
        
        https://github.com/zero01101/openOutpaint-webUI-extension
        
        Tips
        
        Get image prompt
        
        You can drag and drop an image of civit ai into the pnginfo tab and then clic send to txt2img, you need to select the checkpoint model and that is all, everything else is set
        
        Invoke 3.0
        
        https://github.com/invoke-ai/InvokeAI
        
        SD next
        
        https://github.com/vladmandic/automatic
        
        VoltaMl
        
        https://github.com/VoltaML/voltaML-fast-stable-diffusion
        
        Fooocus
        
        https://github.com/lllyasviel/Fooocus
        
        Heavy client UI
        
        NMKD
        
        https://nmkd.itch.io/t2i-gui
        
        Node base ui
        
        ComfyUi
        
        https://github.com/comfyanonymous/ComfyUI
        
        Front end for comfyUi
        
        https://github.com/Stability-AI/StableSwarmUI
        
        /watch?v=M-C5eeDN7Ew
        
        Install
        
        install : /watch?v=Z0BYc-w-jaI
        
        SVD workflow
        
        https://comfyanonymous.github.io/ComfyUI_examples/video/
        
        Download json workflow and drag and drop it inside comfyUI
        
        tuto
        
        adding nodes
        
        right clic > add node
        
        adding custom node manager
        
        into ComfyUi\custom_nodes open a cmd and do git clone https://github.com/ltdrdata/ComfyUI-Manager.git
        
        /watch?v=AbB33AxrcZo
        
        minimize node
        
        right_clic on node > collapse
        
        watch?v=gE_pMdJvAD0
        
        change model path : extra_model_path.yml (point to those of automatic1111 for example)
        
        persitent faces
        
        watch?v=wMLiGhogOPE
        
        ai node engiine
        
        https://github.com/XmYx/ainodes-engine
        
        package manager
        
        https://github.com/LykosAI/StabilityMatrix
      - Training models
        
        Textual inversion (embeddings)
        
        Textual inversions are tiny (about 100 KBs), but you can’t do as much, it must be used with a model checkpoint file
        
        Lora
        
        Bigger than textual inversion but produces better result, not as good as dreambooth
        
        training a lora :/watch?v=N_zhQSx2Q3c
        
        LCM lora
        
        Basically lora but generate images faster, allowing real time render : /watch?v=fotVrleRgi4
        
        Dreambooth
        
        Good result, large models
        
        train dreambooth : /watch?v=usgqmQ0Mq7g
        
        https://github.com/JoePenna/Dreambooth-Stable-Diffusion
        
        ZipLora
        
        Merge various Loras to get specifiq clothes with a certain style for example
        
        https://www.reddit.com/r/StableDiffusion/comments/182f9ej/ziplora_any_subject_in_any_style_by_effectively/
        
        LyCORIS
        
        Alternative to Lora
        
        LyCORIS and LoRA models aim to make minor adjustments to a Stable Diffusion model using a small file. Both modify the U-Net through matrix decomposition, but their approaches differ
      - Cool projects using SD
        
        https://github.com/brycedrennan/imaginAIry python generation
        
        https://github.com/flowtyone/flowty-realtime-lcm-canvas : realtime drawing
        
        https://www.reddit.com/r/StableDiffusion/comments/17flkrm/sd_mangacomics_creation_web_app/
        
        https://github.com/SUDO-AI-3D/zero123plus#zero123-a-single-image-to-consistent-multi-view-diffusion-base-model : multi view from single image
        
        https://ximinng.github.io/DiffSketcher-project/?ref=aiartweekly vector graphics generation
        
        drag diffusion : https://github.com/Yujun-Shi/DragDiffusion
        
        unicontrol (they say better than controlnets) https://github.com/salesforce/UniControl
        
        OneTrainer : https://github.com/Nerogar/OneTrainer
        
        cheat sheet : https://github.com/SupaGruen/StableDiffusion-CheatSheet/releases/tag/v1.2.0
        
        other cheat sheets : https://gist.github.com/SMUsamaShah/218e602d508e891a123929ce7d5885d4
        
        https://github.com/lkwq007/stablediffusion-infinity (outpainting)
      - models
        
        stable diffusion 1.5
        
        1st version, the one we all know
        
        SDXL
        
        new version composed of 2 models, the base and refiner, produces better images
      - inner working
        
        Sampler
        
        what is it?
        
        To produce an image, Stable Diffusion first generates a completely random image in the latent space. The noise predictor then estimates the noise of the image. The predicted noise is subtracted from the image. This process is repeated a dozen times. In the end, you get a clean image.
        This denoising process is called sampling because Stable Diffusion generates a new sample image in each step. The method used in sampling is called the sampler or sampling method.
        
        how to choose?
        
        you need to experiment
        
        https://stable-diffusion-art.com/samplers/
        
        VAE
        
        Improve generated images
        
        https://stable-diffusion-art.com/how-to-use-vae/
        
        You don’t need to use a VAE if you are happy with the result you are getting.
        
        CFG scale
        
        Classifier Free Guidance scale
        
        controls how closely Stable Diffusion should follow your text prompt
        
        Number of step
        
        Depend on the sampler, you need to experiment
      - Prompt examples
        
        https://prompthero.com/stable-diffusion-prompts
        
        https://civitai.com/images
        
        https://arthub.ai/
        
        https://playgroundai.com/feed
  - - - Access via chatGPT
    - - access via discord
    - - they have their own website
  - - - https://discord.gg/stablediffusion
    - - https://discord.gg/unstablediffusion
  - - - https://omriavrahami.com/the-chosen-one/
  - - - allow you to draw a "squeleton pose" in addition to your text prompt and the character in your image will take thtat pose
    - - replace something inside an image
        
        you need to use a special model for inpainting that matches your model
        You normally don’t need an inpainting model when fixing a small patch. They are for inpainting big areas. Since changing the checkpoint model could greatly impact the style, you should use an inpainting model that matches your original model. For example,
        Realistic Vision v5.1 inpainting
        Realistic Vision v5.1
    - - Draw what is outside the image
- - - - gice it 3-5 10 seconds sample of a voice and it will speak with that voice, the voice is slightly different each time you make it speak tho
      - watch?v=OiMRlqcgDL0
      - watch?v=4-2Jk8muo7c
      - watch?v=lZkQUOpLg6g
  - - - train a model with someone voice and then you can speak with that voice using a microphone
  - - - text to music / convert music to other style
      - musicgen audicoraft : https://github.com/facebookresearch/audiocraft
      - /watch?v=la2fGS0dW98
    - - https://github.com/suno-ai/bark
- - - - Running local LLM models
        
        graphical interfaces
        
        Local web servers
        
        LM studio
        
        text-gen oobabooga
        
        https://github.com/oobabooga/text-generation-webui
        
        h2oGPT
        
        Private gpt
        
        Models
        
        Uncensored models
        
        WizardLM 13b : https://huggingface.co/ehartford/WizardLM-13B-Uncensored
        
        Wizard-Vicuna : https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored-GPTQ
        
        dolphin mistral 7b : https://huggingface.co/ehartford/dolphin-2.2-mistral-7b
        
        Nous hermes 13b : https://huggingface.co/NousResearch/Nous-Hermes-13b
        
        Uncensor any model : /watch?v=kta1D5CFHp0
        
        Falcon 40b
        
        good models
        
        near gpt4 : https://huggingface.co/llm-agents/tora-code-13b-v1.0
        
        Coding models
        
        WizardCoder-34B
        
        https://github.com/nlpxucan/WizardLM/tree/main/WizardCoder
        
        /watch?v=tZXrxNIjUiM
        
        LLaVa : a LLM with vision
        
        watch?v=tuV3GQTIy_8
        
        https://github.com/camenduru/LLaVA-colab
    - - ChatGpt (open ai)
      - Grok (twitter)
      - bard (microsoft)
    - - generative agents : https://github.com/joonspk-research/generative_agents
      - AutoGen
        
        tuto : /watch?v=V2qZ_lgxTzg
        
        github : https://github.com/microsoft/autogen
    - - RAG
        
        "teach" document to your model, making it more intelligent
        
        how it works?
        
        embeddings
        
        that is how embedding are used, you make a database of embedding and when a user do a query you search which embedding could be use and add them to the context windows
        
        PDF document are converted to vectors, they are all the same size (like hashes) but the vector allow you to know what is inside the document. so you can turn multiple pdf into embedding, give it to the LLM to make it somewhat more intelligent
        
        where/what to use?
        
        For DEV langchain is a framework to do RAG
        
        PrivateGPT doesIt
        
        h2oGPT does it
        
        It's not as cool a they say, I tested it, the model is not more intelligent, it feels like it does CTRL+F in your documents and uses part of them without understanding what is inside. the awnser were actually worse than without RAG in my case
      - Fine tuning
        
        retrain the model to make it behave differently, get a model that talk like elon musk for example
      - Unscensor any model
        
        Find an interface that can make you choose how the anwser will begin.
        
        make the awnser begin by "Sure, here how to do it :"
        
        Since LLMs are text predictors, autocomplete on steroid, it will bypass the censoring lol