AI

Platform

• https://replicate.com/

• Hugging Face: https://huggingface.co/

  • quicktour: docs/transformers/quicktour

• modal: https://modal.com/

Website

• Chatbot Arena:

  • https://chat.lmsys.org/
  • https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard

• poe: https://poe.com/

  • https://developer.poe.com/server-bots/quick-start

• deepseek: https://chat.deepseek.com/

• Perplexity: https://labs.perplexity.ai/

  • API: https://docs.perplexity.ai/docs/getting-started

Project

• ollama: https://github.com/ollama/ollama

• FastChat: https://github.com/lm-sys/FastChat

• llama: https://ai.meta.com/llama/

• llama Github: https://github.com/facebookresearch/llama

• llama.cpp Port of Facebook's LLaMA model in C/C++

  • https://github.com/ggerganov/llama.cpp

  • Example https://github.com/ggerganov/llama.cpp/blob/master/examples/main/README.md

• llama-gpt

  • https://github.com/getumbrel/llama-gpt

• whisper.cpp Port of OpenAI's Whisper model in C/C++

  • https://github.com/ggerganov/whisper.cpp
  • Talk.wsam: https://whisper.ggerganov.com/talk/

llama.cpp: Run on macOS

# use gpu
./main -m ~/Downloads/llama-2-7b-chat.Q4_K_M.gguf --prompt "what is quantum"

# disable Metal gpu
./main -ngl 0  -m ~/Downloads/llama-2-7b-chat.Q4_K_M.gguf --prompt "what is quantum"

• llm: https://github.com/rustformers/llm
• ggml: https://github.com/ggerganov/ggml

Models

Models are usually downloaded from Hugging Face

• https://github.com/oobabooga/text-generation-webui?tab=readme-ov-file#downloading-models

• https://huggingface.co/TheBloke/Llama-2-7B-GGUF

• https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF

• https://huggingface.co/meta-llama/Llama-2-7b-chat

Leaderboard

• https://evalplus.github.io/leaderboard.html

Serving

• vLLM: https://github.com/vllm-project/vllm
• webui: https://github.com/oobabooga/text-generation-webui

Check later

• https://www.aliyun.com/solution/tech-solution/calbadm?spm=5176.28508143.J_4VYgf18xNlTAyFFbOuOQe.41.4a0b154ac9jGYT&scm=20140722.M_10058081._.V_1
• ChatGLM, LangChain