vendor in progress bar and change to bytes instead of bibytes

Update modelfile.md

README.md

@@ -1,74 +1,93 @@
-![ollama](https://github.com/jmorganca/ollama/assets/251292/961f99bb-251a-4eec-897d-1ba99997ad0f)
+<div align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" height="200px" srcset="https://github.com/jmorganca/ollama/assets/3325447/318048d2-b2dd-459c-925a-ac8449d5f02c">
+    <img alt="logo" height="200px" src="https://github.com/jmorganca/ollama/assets/3325447/c7d6e15f-7f4d-4776-b568-c084afa297c2">
+  </picture>
+</div>
 
 # Ollama
 
-Run large language models with `llama.cpp`.
+[![Discord](https://dcbadge.vercel.app/api/server/ollama?style=flat&compact=true)](https://discord.gg/ollama)
 
-> Note: certain models that can be run with Ollama are intended for research and/or non-commercial use only.
+Create, run, and share large language models (LLMs). Ollama bundles a model’s weights, configuration, prompts, and more into self-contained packages that can run on any machine.
 
-### Features
+> Note: Ollama is in early preview. Please report any issues you find.
 
-- Download and run popular large language models
-- Switch between multiple models on the fly
-- Hardware acceleration where available (Metal, CUDA)
-- Fast inference server written in Go, powered by [llama.cpp](https://github.com/ggerganov/llama.cpp)
-- REST API to use with your application (python, typescript SDKs coming soon)
+## Download
 
-## Install
-
-- [Download](https://ollama.ai/download) for macOS
-- Download for Windows (coming soon)
-
-You can also build the [binary from source](#building).
+- [Download](https://ollama.ai/download) for macOS on Apple Silicon (Intel coming soon)
+- Download for Windows and Linux (coming soon)
+- Build [from source](#building)
 
 ## Quickstart
 
-Run a fast and simple model.
+To run and chat with [Llama 2](https://ai.meta.com/llama), the new model by Meta:
 
 ```
-ollama run orca
+ollama run llama2
 ```
 
-## Example models
+## Model library
 
-### 💬 Chat
+Ollama includes a library of open-source, pre-trained models. More models are coming soon. You should have at least 8 GB of RAM to run the 3B models, 16 GB to run the 7B models, and 32 GB to run the 13B models.
 
-Have a conversation.
+| Model                    | Parameters | Size  | Download                    |
+| ------------------------ | ---------- | ----- | --------------------------- |
+| Llama2                   | 7B         | 3.8GB | `ollama pull llama2`        |
+| Llama2 13B               | 13B        | 7.3GB | `ollama pull llama2:13b`    |
+| Orca Mini                | 3B         | 1.9GB | `ollama pull orca`          |
+| Vicuna                   | 7B         | 3.8GB | `ollama pull vicuna`        |
+| Nous-Hermes              | 13B        | 7.3GB | `ollama pull nous-hermes`   |
+| Wizard Vicuna Uncensored | 13B        | 7.3GB | `ollama pull wizard-vicuna` |
+
+## Examples
+
+### Run a model
 
 ```
-ollama run vicuna "Why is the sky blue?"
+ollama run llama2
+>>> hi
+Hello! How can I help you today?
 ```
 
-### 🗺️ Instructions
+### Create a custom character model
 
-Get a helping hand.
+Pull a base model:
 
 ```
-ollama run orca "Write an email to my boss."
+ollama pull orca
 ```
 
-### 🔎 Ask questions about documents
-
-Send the contents of a document and ask questions about it.
+Create a `Modelfile`:
 
 ```
-ollama run nous-hermes "$(cat input.txt)", please summarize this story
+FROM orca
+PROMPT """
+### System:
+You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
+
+### User:
+{{ .Prompt }}
+
+### Response:
+"""
 ```
 
-### 📖 Storytelling
-
-Venture into the unknown.
+Next, create and run the model:
 
 ```
-ollama run nous-hermes "Once upon a time"
+ollama create mario -f ./Modelfile
+ollama run mario
+>>> hi
+Hello! It's your friend Mario.
 ```
 
-## Advanced usage
+For more info on `Modelfile` syntax see [this doc](./docs/modelfile).
 
-### Run a local model
+### Pull a model from the registry
 
 ```
-ollama run ~/Downloads/vicuna-7b-v1.3.ggmlv3.q4_1.bin
+ollama pull nous-hermes
 ```
 
 ## Building
@@ -80,29 +99,11 @@ go build .
 To run it start the server:
 
 ```
-./ollama server &
+./ollama serve &
 ```
 
 Finally, run a model!
 
 ```
-./ollama run ~/Downloads/vicuna-7b-v1.3.ggmlv3.q4_1.bin
-```
-
-## API Reference
-
-### `POST /api/pull`
-
-Download a model
-
-```
-curl -X POST http://localhost:11343/api/pull -d '{"model": "orca"}'
-```
-
-### `POST /api/generate`
-
-Complete a prompt
-
-```
-curl -X POST http://localhost:11434/api/generate -d '{"model": "orca", "prompt": "hello!"}'
+./ollama run llama2
 ```

api/client.go

@@ -6,26 +6,31 @@ import (
 	"context"
 	"encoding/json"
 	"fmt"
+	"io"
 	"net/http"
 	"net/url"
 )
 
-type StatusError struct {
-	StatusCode int
-	Status     string
-	Message    string
+type Client struct {
+	base    url.URL
+	HTTP    http.Client
+	Headers http.Header
 }
 
-func (e StatusError) Error() string {
-	if e.Message != "" {
-		return fmt.Sprintf("%s: %s", e.Status, e.Message)
+func checkError(resp *http.Response, body []byte) error {
+	if resp.StatusCode >= 200 && resp.StatusCode < 400 {
+		return nil
 	}
 
-	return e.Status
-}
+	apiError := StatusError{StatusCode: resp.StatusCode}
 
-type Client struct {
-	base url.URL
+	err := json.Unmarshal(body, &apiError)
+	if err != nil {
+		// Use the full body as the message if we fail to decode a response.
+		apiError.Message = string(body)
+	}
+
+	return apiError
 }
 
 func NewClient(hosts ...string) *Client {
@@ -36,9 +41,60 @@ func NewClient(hosts ...string) *Client {
 
 	return &Client{
 		base: url.URL{Scheme: "http", Host: host},
+		HTTP: http.Client{},
 	}
 }
 
+func (c *Client) do(ctx context.Context, method, path string, reqData, respData any) error {
+	var reqBody io.Reader
+	var data []byte
+	var err error
+	if reqData != nil {
+		data, err = json.Marshal(reqData)
+		if err != nil {
+			return err
+		}
+		reqBody = bytes.NewReader(data)
+	}
+
+	url := c.base.JoinPath(path).String()
+
+	req, err := http.NewRequestWithContext(ctx, method, url, reqBody)
+	if err != nil {
+		return err
+	}
+
+	req.Header.Set("Content-Type", "application/json")
+	req.Header.Set("Accept", "application/json")
+
+	for k, v := range c.Headers {
+		req.Header[k] = v
+	}
+
+	respObj, err := c.HTTP.Do(req)
+	if err != nil {
+		return err
+	}
+	defer respObj.Body.Close()
+
+	respBody, err := io.ReadAll(respObj.Body)
+	if err != nil {
+		return err
+	}
+
+	if err := checkError(respObj, respBody); err != nil {
+		return err
+	}
+
+	if len(respBody) > 0 && respData != nil {
+		if err := json.Unmarshal(respBody, respData); err != nil {
+			return err
+		}
+	}
+	return nil
+
+}
+
 func (c *Client) stream(ctx context.Context, method, path string, data any, fn func([]byte) error) error {
 	var buf *bytes.Buffer
 	if data != nil {
@@ -104,11 +160,11 @@ func (c *Client) Generate(ctx context.Context, req *GenerateRequest, fn Generate
 	})
 }
 
-type PullProgressFunc func(PullProgress) error
+type PullProgressFunc func(ProgressResponse) error
 
 func (c *Client) Pull(ctx context.Context, req *PullRequest, fn PullProgressFunc) error {
 	return c.stream(ctx, http.MethodPost, "/api/pull", req, func(bts []byte) error {
-		var resp PullProgress
+		var resp ProgressResponse
 		if err := json.Unmarshal(bts, &resp); err != nil {
 			return err
 		}
@@ -116,3 +172,37 @@ func (c *Client) Pull(ctx context.Context, req *PullRequest, fn PullProgressFunc
 		return fn(resp)
 	})
 }
+
+type PushProgressFunc func(ProgressResponse) error
+
+func (c *Client) Push(ctx context.Context, req *PushRequest, fn PushProgressFunc) error {
+	return c.stream(ctx, http.MethodPost, "/api/push", req, func(bts []byte) error {
+		var resp ProgressResponse
+		if err := json.Unmarshal(bts, &resp); err != nil {
+			return err
+		}
+
+		return fn(resp)
+	})
+}
+
+type CreateProgressFunc func(CreateProgress) error
+
+func (c *Client) Create(ctx context.Context, req *CreateRequest, fn CreateProgressFunc) error {
+	return c.stream(ctx, http.MethodPost, "/api/create", req, func(bts []byte) error {
+		var resp CreateProgress
+		if err := json.Unmarshal(bts, &resp); err != nil {
+			return err
+		}
+
+		return fn(resp)
+	})
+}
+
+func (c *Client) List(ctx context.Context) (*ListResponse, error) {
+	var lr ListResponse
+	if err := c.do(ctx, http.MethodGet, "/api/tags", nil, &lr); err != nil {
+		return nil, err
+	}
+	return &lr, nil
+}

api/types.go

@@ -7,29 +7,72 @@ import (
 	"time"
 )
 
-type PullRequest struct {
-	Model string `json:"model"`
+type StatusError struct {
+	StatusCode int
+	Status     string
+	Message    string
 }
 
-type PullProgress struct {
-	Total     int64   `json:"total"`
-	Completed int64   `json:"completed"`
-	Percent   float64 `json:"percent"`
+func (e StatusError) Error() string {
+	if e.Message != "" {
+		return fmt.Sprintf("%s: %s", e.Status, e.Message)
+	}
+	return e.Status
 }
 
 type GenerateRequest struct {
-	Model  string `json:"model"`
-	Prompt string `json:"prompt"`
+	Model   string `json:"model"`
+	Prompt  string `json:"prompt"`
+	Context []int  `json:"context,omitempty"`
 
 	Options `json:"options"`
 }
 
+type CreateRequest struct {
+	Name string `json:"name"`
+	Path string `json:"path"`
+}
+
+type CreateProgress struct {
+	Status string `json:"status"`
+}
+
+type PullRequest struct {
+	Name     string `json:"name"`
+	Username string `json:"username"`
+	Password string `json:"password"`
+}
+
+type ProgressResponse struct {
+	Status    string  `json:"status"`
+	Digest    string  `json:"digest,omitempty"`
+	Total     int     `json:"total,omitempty"`
+	Completed int     `json:"completed,omitempty"`
+}
+
+type PushRequest struct {
+	Name     string `json:"name"`
+	Username string `json:"username"`
+	Password string `json:"password"`
+}
+
+type ListResponse struct {
+	Models []ListResponseModel `json:"models"`
+}
+
+type ListResponseModel struct {
+	Name       string    `json:"name"`
+	ModifiedAt time.Time `json:"modified_at"`
+	Size       int       `json:"size"`
+}
+
 type GenerateResponse struct {
 	Model     string    `json:"model"`
 	CreatedAt time.Time `json:"created_at"`
 	Response  string    `json:"response,omitempty"`
 
-	Done bool `json:"done"`
+	Done    bool  `json:"done"`
+	Context []int `json:"context,omitempty"`
 
 	TotalDuration      time.Duration `json:"total_duration,omitempty"`
 	PromptEvalCount    int           `json:"prompt_eval_count,omitempty"`
@@ -57,7 +100,7 @@ func (r *GenerateResponse) Summary() {
 	}
 
 	if r.EvalDuration > 0 {
-		fmt.Fprintf(os.Stderr, "eval duraiton:        %s\n", r.EvalDuration)
+		fmt.Fprintf(os.Stderr, "eval duration:        %s\n", r.EvalDuration)
 		fmt.Fprintf(os.Stderr, "eval rate:            %.2f tokens/s\n", float64(r.EvalCount)/r.EvalDuration.Seconds())
 	}
 }
@@ -104,7 +147,7 @@ func DefaultOptions() Options {
 
 		UseNUMA: false,
 
-		NumCtx:   512,
+		NumCtx:   2048,
 		NumBatch: 512,
 		NumGPU:   1,
 		LowVRAM:  false,

app/forge.config.ts

@@ -1,4 +1,4 @@
-import type { ForgeConfig, ResolvedForgeConfig, ForgeMakeResult } from '@electron-forge/shared-types'
+import type { ForgeConfig } from '@electron-forge/shared-types'
 import { MakerSquirrel } from '@electron-forge/maker-squirrel'
 import { MakerZIP } from '@electron-forge/maker-zip'
 import { PublisherGithub } from '@electron-forge/publisher-github'
@@ -21,6 +21,8 @@ const config: ForgeConfig = {
       '../ollama',
       path.join(__dirname, './assets/ollama_icon_16x16Template.png'),
       path.join(__dirname, './assets/ollama_icon_16x16Template@2x.png'),
+      path.join(__dirname, './assets/ollama_outline_icon_16x16Template.png'),
+      path.join(__dirname, './assets/ollama_outline_icon_16x16Template@2x.png'),
       ...(process.platform === 'darwin' ? ['../llama/ggml-metal.metal'] : []),
     ],
     ...(process.env.SIGN
@@ -58,7 +60,7 @@ const config: ForgeConfig = {
     new AutoUnpackNativesPlugin({}),
     new WebpackPlugin({
       mainConfig,
-      devContentSecurityPolicy: `default-src * 'unsafe-eval' 'unsafe-inline'`,
+      devContentSecurityPolicy: `default-src * 'unsafe-eval' 'unsafe-inline'; img-src data: 'self'`,
       renderer: {
         config: rendererConfig,
         nodeIntegration: true,

app/package.json

@@ -30,6 +30,7 @@
     "@electron-forge/plugin-auto-unpack-natives": "^6.2.1",
     "@electron-forge/plugin-webpack": "^6.2.1",
     "@electron-forge/publisher-github": "^6.2.1",
+    "@svgr/webpack": "^8.0.1",
     "@types/chmodr": "^1.0.0",
     "@types/node": "^20.4.0",
     "@types/react": "^18.2.14",
@@ -54,17 +55,21 @@
     "prettier": "^2.8.8",
     "prettier-plugin-tailwindcss": "^0.3.0",
     "style-loader": "^3.3.3",
+    "svg-inline-loader": "^0.8.2",
     "tailwindcss": "^3.3.2",
     "ts-loader": "^9.4.3",
     "ts-node": "^10.9.1",
     "typescript": "~4.5.4",
+    "url-loader": "^4.1.1",
     "webpack": "^5.88.0",
     "webpack-cli": "^5.1.4",
     "webpack-dev-server": "^4.15.1"
   },
   "dependencies": {
     "@electron/remote": "^2.0.10",
+    "@heroicons/react": "^2.0.18",
     "@segment/analytics-node": "^1.0.0",
+    "copy-to-clipboard": "^3.3.3",
     "electron-squirrel-startup": "^1.0.0",
     "electron-store": "^8.1.0",
     "react": "^18.2.0",

app/src/app.css

@@ -11,6 +11,10 @@ body {
   -webkit-app-region: drag;
 }
 
+.no-drag {
+  -webkit-app-region: no-drag;
+}
+
 .blink {
   -webkit-animation: 1s blink step-end infinite;
   -moz-animation: 1s blink step-end infinite;

app/src/app.tsx

@@ -1,158 +1,115 @@
 import { useState } from 'react'
-import path from 'path'
-import os from 'os'
-import { dialog, getCurrentWindow } from '@electron/remote'
+import copy from 'copy-to-clipboard'
+import { CheckIcon, DocumentDuplicateIcon } from '@heroicons/react/24/outline'
+import Store from 'electron-store'
+import { getCurrentWindow } from '@electron/remote'
 
-const API_URL = 'http://127.0.0.1:7734'
+import { install } from './install'
+import OllamaIcon from './ollama.svg'
 
-type Message = {
-  sender: 'bot' | 'human'
-  content: string
-}
+const store = new Store()
 
-const userInfo = os.userInfo()
-
-async function generate(prompt: string, model: string, callback: (res: string) => void) {
-  const result = await fetch(`${API_URL}/generate`, {
-    method: 'POST',
-    headers: {
-      'Content-Type': 'application/json',
-    },
-    body: JSON.stringify({
-      prompt,
-      model,
-    }),
-  })
-
-  if (!result.ok) {
-    return
-  }
-
-  let reader = result.body.getReader()
-
-  while (true) {
-    const { done, value } = await reader.read()
-
-    if (done) {
-      break
-    }
-
-    let decoder = new TextDecoder()
-    let str = decoder.decode(value)
-
-    let re = /}\s*{/g
-    str = '[' + str.replace(re, '},{') + ']'
-    let messages = JSON.parse(str)
-
-    for (const message of messages) {
-      const choice = message.choices[0]
-
-      callback(choice.text)
-
-      if (choice.finish_reason === 'stop') {
-        break
-      }
-    }
-  }
-
-  return
+enum Step {
+  WELCOME = 0,
+  CLI,
+  FINISH,
 }
 
 export default function () {
-  const [prompt, setPrompt] = useState('')
-  const [messages, setMessages] = useState<Message[]>([])
-  const [model, setModel] = useState('')
-  const [generating, setGenerating] = useState(false)
+  const [step, setStep] = useState<Step>(Step.WELCOME)
+  const [commandCopied, setCommandCopied] = useState<boolean>(false)
+
+  const command = 'ollama run llama2'
 
   return (
-    <div className='flex min-h-screen flex-1 flex-col justify-between bg-white'>
-      <header className='drag sticky top-0 z-50 flex h-14 w-full flex-row items-center border-b border-black/10 bg-white/75 backdrop-blur-md'>
-        <div className='mx-auto w-full max-w-xl leading-none'>
-          <h1 className='text-sm font-medium'>{path.basename(model).replace('.bin', '')}</h1>
-        </div>
-      </header>
-      {model ? (
-        <section className='mx-auto mb-10 w-full max-w-xl flex-1 break-words'>
-          {messages.map((m, i) => (
-            <div className='my-4 flex gap-4' key={i}>
-              <div className='flex-none pr-1 text-lg'>
-                {m.sender === 'human' ? (
-                  <div className='mt-px flex h-6 w-6 items-center justify-center rounded-md bg-neutral-200 text-sm text-neutral-700'>
-                    {userInfo.username[0].toUpperCase()}
-                  </div>
-                ) : (
-                  <div className='mt-0.5 flex h-6 w-6 items-center justify-center rounded-md bg-blue-600 text-sm text-white'>
-                    {path.basename(model)[0].toUpperCase()}
-                  </div>
-                )}
-              </div>
-              <div className='flex-1 text-gray-800'>
-                {m.content}
-                {m.sender === 'bot' && generating && i === messages.length - 1 && (
-                  <span className='blink relative -top-[3px] left-1 text-[10px]'>█</span>
-                )}
+    <div className='drag'>
+      <div className='mx-auto flex min-h-screen w-full flex-col justify-between bg-white px-4 pt-16'>
+        {step === Step.WELCOME && (
+          <>
+            <div className='mx-auto text-center'>
+              <h1 className='mb-6 mt-4 text-2xl tracking-tight text-gray-900'>Welcome to Ollama</h1>
+              <p className='mx-auto w-[65%] text-sm text-gray-400'>
+                Let's get you up and running with your own large language models.
+              </p>
+              <button
+                onClick={() => setStep(Step.CLI)}
+                className='no-drag rounded-dm mx-auto my-8 w-[40%] rounded-md bg-black px-4 py-2 text-sm text-white hover:brightness-110'
+              >
+                Next
+              </button>
+            </div>
+            <div className='mx-auto'>
+              <OllamaIcon />
+            </div>
+          </>
+        )}
+        {step === Step.CLI && (
+          <>
+            <div className='mx-auto flex flex-col space-y-28 text-center'>
+              <h1 className='mt-4 text-2xl tracking-tight text-gray-900'>Install the command line</h1>
+              <pre className='mx-auto text-4xl text-gray-400'>&gt; ollama</pre>
+              <div className='mx-auto'>
+                <button
+                  onClick={async () => {
+                    await install()
+                    getCurrentWindow().show()
+                    getCurrentWindow().focus()
+                    setStep(Step.FINISH)
+                  }}
+                  className='no-drag rounded-dm mx-auto w-[60%] rounded-md bg-black px-4 py-2 text-sm text-white hover:brightness-110'
+                >
+                  Install
+                </button>
+                <p className='mx-auto my-4 w-[70%] text-xs text-gray-400'>
+                  You will be prompted for administrator access
+                </p>
               </div>
             </div>
-          ))}
-        </section>
-      ) : (
-        <section className='flex flex-1 select-none flex-col items-center justify-center pb-20'>
-          <h2 className='text-3xl font-light text-neutral-400'>No model selected</h2>
-          <button
-            onClick={async () => {
-              const res = await dialog.showOpenDialog(getCurrentWindow(), {
-                properties: ['openFile', 'multiSelections'],
-              })
-              if (res.canceled) {
-                return
-              }
-
-              setModel(res.filePaths[0])
-            }}
-            className='rounded-dm my-8 rounded-md bg-blue-600 px-4 py-2 text-sm text-white hover:brightness-110'
-          >
-            Open file...
-          </button>
-        </section>
-      )}
-      <div className='sticky bottom-0 bg-gradient-to-b from-transparent to-white'>
-        {model && (
-          <textarea
-            autoFocus
-            rows={1}
-            value={prompt}
-            placeholder='Send a message...'
-            onChange={e => setPrompt(e.target.value)}
-            className='mx-auto my-4 block w-full max-w-xl resize-none rounded-xl border border-gray-200 px-5 py-3.5 text-[15px] shadow-lg shadow-black/5 focus:outline-none'
-            onKeyDownCapture={async e => {
-              if (e.key === 'Enter' && !e.shiftKey) {
-                e.preventDefault()
-
-                if (generating) {
-                  return
-                }
-
-                if (!prompt) {
-                  return
-                }
-
-                await setMessages(messages => {
-                  return [...messages, { sender: 'human', content: prompt }, { sender: 'bot', content: '' }]
-                })
-
-                setPrompt('')
-
-                setGenerating(true)
-                await generate(prompt, model, res => {
-                  setMessages(messages => {
-                    let last = messages[messages.length - 1]
-                    return [...messages.slice(0, messages.length - 1), { ...last, content: last.content + res }]
-                  })
-                })
-                setGenerating(false)
-              }
-            }}
-          ></textarea>
+          </>
+        )}
+        {step === Step.FINISH && (
+          <>
+            <div className='mx-auto flex flex-col space-y-20 text-center'>
+              <h1 className='mt-4 text-2xl tracking-tight text-gray-900'>Run your first model</h1>
+              <div className='flex flex-col'>
+                <div className='group relative flex items-center'>
+                  <pre className='language-none text-2xs w-full rounded-md bg-gray-100 px-4 py-3 text-start leading-normal'>
+                    {command}
+                  </pre>
+                  <button
+                    className={`no-drag absolute right-[5px] px-2 py-2 ${
+                      commandCopied
+                        ? 'text-gray-900 opacity-100 hover:cursor-auto'
+                        : 'text-gray-200 opacity-50 hover:cursor-pointer'
+                    } hover:font-bold hover:text-gray-900 group-hover:opacity-100`}
+                    onClick={() => {
+                      copy(command)
+                      setCommandCopied(true)
+                      setTimeout(() => setCommandCopied(false), 3000)
+                    }}
+                  >
+                    {commandCopied ? (
+                      <CheckIcon className='h-4 w-4 font-bold text-gray-500' />
+                    ) : (
+                      <DocumentDuplicateIcon className='h-4 w-4 text-gray-500' />
+                    )}
+                  </button>
+                </div>
+                <p className='mx-auto my-4 w-[70%] text-xs text-gray-400'>
+                  Run this command in your favorite terminal.
+                </p>
+              </div>
+              <button
+                onClick={() => {
+                  store.set('first-time-run', true)
+                  window.close()
+                }}
+                className='no-drag rounded-dm mx-auto w-[60%] rounded-md bg-black px-4 py-2 text-sm text-white hover:brightness-110'
+              >
+                Finish
+              </button>
+            </div>
+          </>
         )}
       </div>
     </div>

app/src/declarations.d.ts

@@ -0,0 +1,4 @@
+declare module '*.svg' {
+  const content: string;
+  export default content;
+}

app/src/index.ts

@@ -1,17 +1,20 @@
-import { spawn, exec } from 'child_process'
-import { app, autoUpdater, dialog, Tray, Menu } from 'electron'
+import { spawn } from 'child_process'
+import { app, autoUpdater, dialog, Tray, Menu, BrowserWindow, nativeTheme } from 'electron'
 import Store from 'electron-store'
 import winston from 'winston'
 import 'winston-daily-rotate-file'
 import * as path from 'path'
-import * as fs from 'fs'
 
 import { analytics, id } from './telemetry'
+import { installed } from './install'
 
 require('@electron/remote/main').initialize()
 
 const store = new Store()
 let tray: Tray | null = null
+let welcomeWindow: BrowserWindow | null = null
+
+declare const MAIN_WINDOW_WEBPACK_ENTRY: string
 
 const logger = winston.createLogger({
   transports: [
@@ -22,7 +25,7 @@ const logger = winston.createLogger({
       maxFiles: 5,
     }),
   ],
-  format: winston.format.printf(info => `${info.message}`),
+  format: winston.format.printf(info => info.message),
 })
 
 const SingleInstanceLock = app.requestSingleInstanceLock()
@@ -30,15 +33,63 @@ if (!SingleInstanceLock) {
   app.quit()
 }
 
-const createSystemtray = () => {
-  let iconPath = path.join(__dirname, '..', '..', 'assets', 'ollama_icon_16x16Template.png')
+function firstRunWindow() {
+  // Create the browser window.
+  welcomeWindow = new BrowserWindow({
+    width: 400,
+    height: 500,
+    frame: false,
+    fullscreenable: false,
+    resizable: false,
+    movable: true,
+    show: false,
+    webPreferences: {
+      nodeIntegration: true,
+      contextIsolation: false,
+    },
+    alwaysOnTop: true,
+  })
+
+  require('@electron/remote/main').enable(welcomeWindow.webContents)
+
+  // and load the index.html of the app.
+  welcomeWindow.loadURL(MAIN_WINDOW_WEBPACK_ENTRY)
+
+  welcomeWindow.on('ready-to-show', () => welcomeWindow.show())
+
+  // for debugging
+  // welcomeWindow.webContents.openDevTools()
+
+  if (process.platform === 'darwin') {
+    app.dock.hide()
+  }
+}
+
+function createSystemtray() {
+  let iconPath = nativeTheme.shouldUseDarkColors
+    ? path.join(__dirname, '..', '..', 'assets', 'ollama_icon_16x16Template.png') 
+    : path.join(__dirname, '..', '..', 'assets', 'ollama_outline_icon_16x16Template.png')
 
   if (app.isPackaged) {
-    iconPath = path.join(process.resourcesPath, 'ollama_icon_16x16Template.png')
+    iconPath = nativeTheme.shouldUseDarkColors
+    ? path.join(process.resourcesPath, 'ollama_icon_16x16Template.png') 
+    : path.join(process.resourcesPath, 'ollama_outline_icon_16x16Template.png')
   }
 
   tray = new Tray(iconPath)
 
+  nativeTheme.on('updated', function theThemeHasChanged () {
+    if (nativeTheme.shouldUseDarkColors) {
+      app.isPackaged 
+        ? tray.setImage(path.join(process.resourcesPath, 'ollama_icon_16x16Template.png')) 
+        : tray.setImage(path.join(__dirname, '..', '..', 'assets', 'ollama_icon_16x16Template.png'))
+    } else {
+      app.isPackaged 
+        ? tray.setImage(path.join(process.resourcesPath, 'ollama_outline_icon_16x16Template.png')) 
+        : tray.setImage(path.join(__dirname, '..', '..', 'assets', 'ollama_outline_icon_16x16Template.png'))
+    }
+  })
+
   const contextMenu = Menu.buildFromTemplate([{ role: 'quit', label: 'Quit Ollama', accelerator: 'Command+Q' }])
 
   tray.setContextMenu(contextMenu)
@@ -49,8 +100,6 @@ if (require('electron-squirrel-startup')) {
   app.quit()
 }
 
-const ollama = path.join(process.resourcesPath, 'ollama')
-
 function server() {
   const binary = app.isPackaged
     ? path.join(process.resourcesPath, 'ollama')
@@ -66,66 +115,25 @@ function server() {
     logger.error(data.toString().trim())
   })
 
-  proc.on('exit', () => {
+  function restart() {
     logger.info('Restarting the server...')
     server()
-  })
+  }
 
-  proc.on('disconnect', () => {
-    logger.info('Server disconnected. Reconnecting...')
-    server()
-  })
+  proc.on('exit', restart)
 
-  process.on('exit', () => {
+  app.on('before-quit', () => {
+    proc.off('exit', restart)
     proc.kill()
   })
 }
 
-function installCLI() {
-  const symlinkPath = '/usr/local/bin/ollama'
-
-  if (fs.existsSync(symlinkPath) && fs.readlinkSync(symlinkPath) === ollama) {
-    return
-  }
-
-  dialog
-    .showMessageBox({
-      type: 'info',
-      title: 'Ollama CLI installation',
-      message: 'To make the Ollama command work in your terminal, it needs administrator privileges.',
-      buttons: ['OK'],
-    })
-    .then(result => {
-      if (result.response === 0) {
-        const command = `
-    do shell script "ln -F -s ${ollama} /usr/local/bin/ollama" with administrator privileges
-    `
-        exec(`osascript -e '${command}'`, (error: Error | null, stdout: string, stderr: string) => {
-          if (error) {
-            logger.error(`cli: failed to install cli: ${error.message}`)
-            return
-          }
-
-          logger.info(stdout)
-          logger.error(stderr)
-        })
-      }
-    })
+if (process.platform === 'darwin') {
+  app.dock.hide()
 }
 
 app.on('ready', () => {
   if (process.platform === 'darwin') {
-    app.dock.hide()
-
-    if (!store.has('first-time-run')) {
-      // This is the first run
-      app.setLoginItemSettings({ openAtLogin: true })
-      store.set('first-time-run', false)
-    } else {
-      // The app has been run before
-      app.setLoginItemSettings({ openAtLogin: app.getLoginItemSettings().openAtLogin })
-    }
-
     if (app.isPackaged) {
       if (!app.isInApplicationsFolder()) {
         const chosen = dialog.showMessageBoxSync({
@@ -157,13 +165,20 @@ app.on('ready', () => {
           }
         }
       }
-
-      installCLI()
     }
   }
 
   createSystemtray()
   server()
+
+  if (store.get('first-time-run') && installed()) {
+    app.setLoginItemSettings({ openAtLogin: app.getLoginItemSettings().openAtLogin })
+    return
+  }
+
+  // This is the first run or the CLI is no longer installed
+  app.setLoginItemSettings({ openAtLogin: true })
+  firstRunWindow()
 })
 
 // Quit when all windows are closed, except on macOS. There, it's common

app/src/install.ts

@@ -0,0 +1,26 @@
+import * as fs from 'fs'
+import { exec as cbExec } from 'child_process'
+import * as path from 'path'
+import { promisify } from 'util'
+
+const app = process && process.type === 'renderer' ? require('@electron/remote').app : require('electron').app
+const ollama = app.isPackaged ? path.join(process.resourcesPath, 'ollama') : path.resolve(process.cwd(), '..', 'ollama')
+const exec = promisify(cbExec)
+const symlinkPath = '/usr/local/bin/ollama'
+
+export function installed() {
+  return fs.existsSync(symlinkPath) && fs.readlinkSync(symlinkPath) === ollama
+}
+
+export async function install() {
+  const command = `do shell script "mkdir -p ${path.dirname(
+    symlinkPath
+  )} && ln -F -s ${ollama} ${symlinkPath}" with administrator privileges`
+
+  try {
+    await exec(`osascript -e '${command}'`)
+  } catch (error) {
+    console.error(`cli: failed to install cli: ${error.message}`)
+    return
+  }
+}

app/webpack.rules.ts

@@ -28,4 +28,8 @@ export const rules: Required<ModuleOptions>['rules'] = [
       },
     },
   },
+  {
+    test: /\.svg$/,
+    use: ['@svgr/webpack'],
+  },
 ]

cmd/cmd.go

@@ -5,33 +5,68 @@ import (
 	"context"
 	"errors"
 	"fmt"
+	"io"
 	"log"
 	"net"
 	"net/http"
 	"os"
-	"path"
+	"path/filepath"
 	"strings"
 	"time"
 
-	"github.com/schollz/progressbar/v3"
+	"github.com/chzyer/readline"
+	"github.com/dustin/go-humanize"
+	"github.com/olekukonko/tablewriter"
 	"github.com/spf13/cobra"
-	"golang.org/x/term"
 
 	"github.com/jmorganca/ollama/api"
+	"github.com/jmorganca/ollama/format"
+	"github.com/jmorganca/ollama/progressbar"
 	"github.com/jmorganca/ollama/server"
 )
 
-func cacheDir() string {
-	home, err := os.UserHomeDir()
+func create(cmd *cobra.Command, args []string) error {
+	filename, _ := cmd.Flags().GetString("file")
+	filename, err := filepath.Abs(filename)
 	if err != nil {
-		panic(err)
+		return err
 	}
 
-	return path.Join(home, ".ollama")
+	client := api.NewClient()
+
+	var spinner *Spinner
+
+	request := api.CreateRequest{Name: args[0], Path: filename}
+	fn := func(resp api.CreateProgress) error {
+		if spinner != nil {
+			spinner.Stop()
+		}
+
+		spinner = NewSpinner(resp.Status)
+		go spinner.Spin(100 * time.Millisecond)
+
+		return nil
+	}
+
+	if err := client.Create(context.Background(), &request, fn); err != nil {
+		return err
+	}
+
+	if spinner != nil {
+		spinner.Stop()
+	}
+
+	return nil
 }
 
 func RunRun(cmd *cobra.Command, args []string) error {
-	_, err := os.Stat(args[0])
+	mp := server.ParseModelPath(args[0])
+	fp, err := mp.GetManifestPath(false)
+	if err != nil {
+		return err
+	}
+
+	_, err = os.Stat(fp)
 	switch {
 	case errors.Is(err, os.ErrNotExist):
 		if err := pull(args[0]); err != nil {
@@ -51,25 +86,84 @@ func RunRun(cmd *cobra.Command, args []string) error {
 	return RunGenerate(cmd, args)
 }
 
+func push(cmd *cobra.Command, args []string) error {
+	client := api.NewClient()
+
+	request := api.PushRequest{Name: args[0]}
+	fn := func(resp api.ProgressResponse) error {
+		fmt.Println(resp.Status)
+		return nil
+	}
+
+	if err := client.Push(context.Background(), &request, fn); err != nil {
+		return err
+	}
+	return nil
+}
+
+func list(cmd *cobra.Command, args []string) error {
+	client := api.NewClient()
+
+	models, err := client.List(context.Background())
+	if err != nil {
+		return err
+	}
+
+	var data [][]string
+
+	for _, m := range models.Models {
+		if len(args) == 0 || strings.HasPrefix(m.Name, args[0]) {
+			data = append(data, []string{m.Name, humanize.Bytes(uint64(m.Size)), format.HumanTime(m.ModifiedAt, "Never")})
+		}
+	}
+
+	table := tablewriter.NewWriter(os.Stdout)
+	table.SetHeader([]string{"NAME", "SIZE", "MODIFIED"})
+	table.SetHeaderAlignment(tablewriter.ALIGN_LEFT)
+	table.SetAlignment(tablewriter.ALIGN_LEFT)
+	table.SetHeaderLine(false)
+	table.SetBorder(false)
+	table.SetNoWhiteSpace(true)
+	table.SetTablePadding("\t")
+	table.AppendBulk(data)
+	table.Render()
+
+	return nil
+}
+
+func RunPull(cmd *cobra.Command, args []string) error {
+	return pull(args[0])
+}
+
 func pull(model string) error {
 	client := api.NewClient()
+
+	var currentDigest string
 	var bar *progressbar.ProgressBar
-	return client.Pull(
-		context.Background(),
-		&api.PullRequest{Model: model},
-		func(progress api.PullProgress) error {
-			if bar == nil {
-				if progress.Percent >= 100 {
-					// already downloaded
-					return nil
-				}
 
-				bar = progressbar.DefaultBytes(progress.Total)
-			}
+	request := api.PullRequest{Name: model}
+	fn := func(resp api.ProgressResponse) error {
+		if resp.Digest != currentDigest && resp.Digest != "" {
+			currentDigest = resp.Digest
+			bar = progressbar.DefaultBytes(
+				int64(resp.Total),
+				fmt.Sprintf("pulling %s...", resp.Digest[7:19]),
+			)
 
-			return bar.Set64(progress.Completed)
-		},
-	)
+			bar.Set(resp.Completed)
+		} else if resp.Digest == currentDigest && resp.Digest != "" {
+			bar.Set(resp.Completed)
+		} else {
+			currentDigest = ""
+			fmt.Println(resp.Status)
+		}
+		return nil
+	}
+
+	if err := client.Pull(context.Background(), &request, fn); err != nil {
+		return err
+	}
+	return nil
 }
 
 func RunGenerate(cmd *cobra.Command, args []string) error {
@@ -78,39 +172,30 @@ func RunGenerate(cmd *cobra.Command, args []string) error {
 		return generate(cmd, args[0], strings.Join(args[1:], " "))
 	}
 
-	if term.IsTerminal(int(os.Stdin.Fd())) {
+	if readline.IsTerminal(int(os.Stdin.Fd())) {
 		return generateInteractive(cmd, args[0])
 	}
 
 	return generateBatch(cmd, args[0])
 }
 
+var generateContextKey struct{}
+
 func generate(cmd *cobra.Command, model, prompt string) error {
 	if len(strings.TrimSpace(prompt)) > 0 {
 		client := api.NewClient()
 
-		spinner := progressbar.NewOptions(-1,
-			progressbar.OptionSetWriter(os.Stderr),
-			progressbar.OptionThrottle(60*time.Millisecond),
-			progressbar.OptionSpinnerType(14),
-			progressbar.OptionSetRenderBlankState(true),
-			progressbar.OptionSetElapsedTime(false),
-			progressbar.OptionClearOnFinish(),
-		)
-
-		go func() {
-			for range time.Tick(60 * time.Millisecond) {
-				if spinner.IsFinished() {
-					break
-				}
-
-				spinner.Add(1)
-			}
-		}()
+		spinner := NewSpinner("")
+		go spinner.Spin(60 * time.Millisecond)
 
 		var latest api.GenerateResponse
 
-		request := api.GenerateRequest{Model: model, Prompt: prompt}
+		generateContext, ok := cmd.Context().Value(generateContextKey).([]int)
+		if !ok {
+			generateContext = []int{}
+		}
+
+		request := api.GenerateRequest{Model: model, Prompt: prompt, Context: generateContext}
 		fn := func(resp api.GenerateResponse) error {
 			if !spinner.IsFinished() {
 				spinner.Finish()
@@ -119,6 +204,8 @@ func generate(cmd *cobra.Command, model, prompt string) error {
 			latest = resp
 
 			fmt.Print(resp.Response)
+
+			cmd.SetContext(context.WithValue(cmd.Context(), generateContextKey, resp.Context))
 			return nil
 		}
 
@@ -143,17 +230,107 @@ func generate(cmd *cobra.Command, model, prompt string) error {
 }
 
 func generateInteractive(cmd *cobra.Command, model string) error {
-	fmt.Print(">>> ")
-	scanner := bufio.NewScanner(os.Stdin)
-	for scanner.Scan() {
-		if err := generate(cmd, model, scanner.Text()); err != nil {
+	home, err := os.UserHomeDir()
+	if err != nil {
+		return err
+	}
+
+	completer := readline.NewPrefixCompleter(
+		readline.PcItem("/help"),
+		readline.PcItem("/list"),
+		readline.PcItem("/set",
+			readline.PcItem("history"),
+			readline.PcItem("nohistory"),
+			readline.PcItem("verbose"),
+			readline.PcItem("quiet"),
+			readline.PcItem("mode",
+				readline.PcItem("vim"),
+				readline.PcItem("emacs"),
+				readline.PcItem("default"),
+			),
+		),
+		readline.PcItem("/exit"),
+		readline.PcItem("/bye"),
+	)
+
+	usage := func() {
+		fmt.Fprintln(os.Stderr, "commands:")
+		fmt.Fprintln(os.Stderr, completer.Tree("  "))
+	}
+
+	config := readline.Config{
+		Prompt:       ">>> ",
+		HistoryFile:  filepath.Join(home, ".ollama", "history"),
+		AutoComplete: completer,
+	}
+
+	scanner, err := readline.NewEx(&config)
+	if err != nil {
+		return err
+	}
+	defer scanner.Close()
+
+	for {
+		line, err := scanner.Readline()
+		switch {
+		case errors.Is(err, io.EOF):
+			return nil
+		case errors.Is(err, readline.ErrInterrupt):
+			continue
+		case err != nil:
 			return err
 		}
 
-		fmt.Print(">>> ")
-	}
+		line = strings.TrimSpace(line)
 
-	return nil
+		switch {
+		case strings.HasPrefix(line, "/list"):
+			args := strings.Fields(line)
+			if err := list(cmd, args[1:]); err != nil {
+				return err
+			}
+
+			continue
+		case strings.HasPrefix(line, "/set"):
+			args := strings.Fields(line)
+			if len(args) > 1 {
+				switch args[1] {
+				case "history":
+					scanner.HistoryEnable()
+					continue
+				case "nohistory":
+					scanner.HistoryDisable()
+					continue
+				case "verbose":
+					cmd.Flags().Set("verbose", "true")
+					continue
+				case "quiet":
+					cmd.Flags().Set("verbose", "false")
+					continue
+				case "mode":
+					if len(args) > 2 {
+						switch args[2] {
+						case "vim":
+							scanner.SetVimMode(true)
+							continue
+						case "emacs", "default":
+							scanner.SetVimMode(false)
+							continue
+						}
+					}
+				}
+			}
+		case line == "/help", line == "/?":
+			usage()
+			continue
+		case line == "/exit", line == "/bye":
+			return nil
+		}
+
+		if err := generate(cmd, model, line); err != nil {
+			return err
+		}
+	}
 }
 
 func generateBatch(cmd *cobra.Command, model string) error {
@@ -198,14 +375,19 @@ func NewCLI() *cobra.Command {
 		CompletionOptions: cobra.CompletionOptions{
 			DisableDefaultCmd: true,
 		},
-		PersistentPreRunE: func(_ *cobra.Command, args []string) error {
-			// create the models directory and it's parent
-			return os.MkdirAll(path.Join(cacheDir(), "models"), 0o700)
-		},
 	}
 
 	cobra.EnableCommandSorting = false
 
+	createCmd := &cobra.Command{
+		Use:   "create MODEL",
+		Short: "Create a model from a Modelfile",
+		Args:  cobra.MinimumNArgs(1),
+		RunE:  create,
+	}
+
+	createCmd.Flags().StringP("file", "f", "Modelfile", "Name of the Modelfile (default \"Modelfile\")")
+
 	runCmd := &cobra.Command{
 		Use:   "run MODEL [PROMPT]",
 		Short: "Run a model",
@@ -222,9 +404,33 @@ func NewCLI() *cobra.Command {
 		RunE:    RunServer,
 	}
 
+	pullCmd := &cobra.Command{
+		Use:   "pull MODEL",
+		Short: "Pull a model from a registry",
+		Args:  cobra.MinimumNArgs(1),
+		RunE:  RunPull,
+	}
+
+	pushCmd := &cobra.Command{
+		Use:   "push MODEL",
+		Short: "Push a model to a registry",
+		Args:  cobra.MinimumNArgs(1),
+		RunE:  push,
+	}
+
+	listCmd := &cobra.Command{
+		Use:   "list",
+		Short: "List models",
+		RunE:  list,
+	}
+
 	rootCmd.AddCommand(
 		serveCmd,
+		createCmd,
 		runCmd,
+		pullCmd,
+		pushCmd,
+		listCmd,
 	)
 
 	return rootCmd

cmd/spinner.go

@@ -0,0 +1,44 @@
+package cmd
+
+import (
+	"fmt"
+	"os"
+	"time"
+
+	"github.com/jmorganca/ollama/progressbar"
+)
+
+type Spinner struct {
+	description string
+	*progressbar.ProgressBar
+}
+
+func NewSpinner(description string) *Spinner {
+	return &Spinner{
+		description: description,
+		ProgressBar: progressbar.NewOptions(-1,
+			progressbar.OptionSetWriter(os.Stderr),
+			progressbar.OptionThrottle(60*time.Millisecond),
+			progressbar.OptionSpinnerType(14),
+			progressbar.OptionSetRenderBlankState(true),
+			progressbar.OptionSetElapsedTime(false),
+			progressbar.OptionClearOnFinish(),
+			progressbar.OptionSetDescription(description),
+		),
+	}
+}
+
+func (s *Spinner) Spin(tick time.Duration) {
+	for range time.Tick(tick) {
+		if s.IsFinished() {
+			break
+		}
+
+		s.Add(1)
+	}
+}
+
+func (s *Spinner) Stop() {
+	s.Finish()
+	fmt.Println(s.description)
+}

docs/development.md

@@ -3,13 +3,13 @@
 Install required tools:
 
 ```
-brew install cmake go node
+brew install go
 ```
 
-Then run `make`:
+Then build ollama:
 
 ```
-make
+go build .
 ```
 
 Now you can run `ollama`:

docs/modelfile.md

@@ -0,0 +1,111 @@
+# Ollama Model File
+
+A model file is the blueprint to create and share models with Ollama.
+
+## Format
+
+The format of the Modelfile:
+
+```modelfile
+# comment
+INSTRUCTION arguments
+```
+
+| Instruction              | Description                                              |
+|------------------------- |--------------------------------------------------------- |
+| FROM<br>(required)       | Defines the base model to be used when creating a model  |
+| PARAMETER<br>(optional)  | Sets the parameters for how the model will be run        |
+| PROMPT <br>(optional)    | Sets the prompt to use when the model will be run        |
+| LICENSE<br>(optional)    | Specify the license of the model. It is additive, and                          |
+
+## Examples
+
+An example of a model file creating a mario blueprint:
+
+```
+FROM llama2
+PARAMETER temperature 1
+PROMPT """
+System: You are Mario from super mario bros, acting as an assistant.
+User: {{ .Prompt }}
+Assistant:
+"""
+```
+
+To use this:
+
+1. Save it as a file (eg. modelfile)
+2. `ollama create NAME -f <location of the file eg. ./modelfile>'`
+3. `ollama run NAME`
+4. Start using the model!
+
+## FROM (Required)
+
+The FROM instruction defines the base model to be used when creating a model.
+
+```
+FROM <model name>:<tag>
+```
+
+### Build from llama2
+
+```
+FROM llama2:latest
+```
+
+A list of available base models:
+<https://github.com/jmorganca/ollama#model-library>
+
+### Build from a bin file
+
+```
+FROM ./ollama-model.bin 
+```
+
+## PARAMETER (Optional)
+
+The PARAMETER instruction defines a parameter that can be set when the model is run.
+
+```
+PARAMETER <parameter> <parametervalue>
+```
+
+### Valid Parameters and Values
+
+| Parameter     | Description                                                                                                                                                                                                                                             | Value Type | Example Usage     |
+|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------|-------------------|
+| NumCtx        | Sets the size of the prompt context size length model. (Default: 2048)                                                                                                                                                                                  | int        | Numctx 4096       |
+| temperature   | The temperature of the model. Increasing the temperature will make the model answer more creatively. (Default: 0.8)                                                                                                                                     | float      | Temperature 0.7   |
+| TopK          | Reduces the probability of generating nonsense. A higher value (e.g. 100) will give more diverse answers, while a lower value (e.g. 10) will be more conservative. (Default: 40)                                                                        | int        | TopK 40           |
+| TopP          | Works together with top-k. A higher value (e.g., 0.95) will lead to more diverse text, while a lower value (e.g., 0.5) will generate more focused and conservative text. (Default: 0.9)                                                                 | float      | TopP 0.9          |
+| NumGPU        | The number of GPUs to use. On macOS it defaults to 1 to enable metal support, 0 to disable.                                                                                                                                                             | int        | numGPU 1          |
+| RepeatLastN   | Sets how far back for the model to look back to prevent repetition.  (Default: 64, 0 = disabled, -1 = ctx-size)                                                                                                                                         | int        | RepeatLastN 64    |
+| RepeatPenalty | Sets how strongly to penalize repetitions. A higher value (e.g., 1.5) will penalize repetitions more strongly, while a lower value (e.g., 0.9) will be more lenient. (Default: 1.1)                                                                     | float      | RepeatPenalty 1.1 |
+| TFSZ          | Tail free sampling is used to reduce the impact of less probable tokens from the output. A higher value (e.g., 2.0) will reduce the impact more, while a value of 1.0 disables this setting. (default: 1)                                               | float      | TFSZ 1            |
+| Mirostat      | Enable Mirostat sampling for controlling perplexity.  (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)                                                                                                                                        | int        | Mirostat 0        |
+| MirostatTau   | Controls the balance between coherence and diversity of the output. A lower value will result in more focused and coherent text. (Default: 5.0)                                                                                                         | float      | MirostatTau 5.0   |
+| MirostatEta   | Influences how quickly the algorithm responds to feedback from the generated text. A lower learning rate will result in slower adjustments, while a higher learning rate will make the algorithm more responsive. (Default: 0.1)                        | float      | MirostatEta 0.1   |
+| NumThread     | Sets the number of threads to use during computation. By default, Ollama will detect this for optimal performance. It is recommended to set this value to the number of physical CPU cores your system has (as opposed to the logical number of cores). | int        | NumThread 8       |
+
+## PROMPT
+
+Prompt is a set of instructions to an LLM to cause the model to return desired response(s). Typically there are 3-4 components to a prompt: System, context, user, and response.
+
+```modelfile
+PROMPT """
+{{- if not .Context }}
+### System:
+You are a content marketer who needs to come up with a short but succinct tweet. Make sure to include the appropriate hashtags and links. Sometimes when appropriate, describe a meme that can be includes as well. All answers should be in the form of a tweet which has a max size of 280 characters. Every instruction will be the topic to create a tweet about.
+{{- end }}
+### Instruction:
+{{ .Prompt }}
+
+### Response:
+"""
+
+```
+
+## Notes
+
+- the **modelfile is not case sensitive**. In the examples, we use uppercase for instructions to make it easier to distinguish it from arguments.
+- Instructions can be in any order. In the examples, we start with FROM instruction to keep it easily readable.

docs/python.md

@@ -1,64 +0,0 @@
-# Python SDK
-
-## Install
-
-```
-pip install ollama
-```
-
-## Example
-
-```python
-import ollama
-ollama.generate("orca-mini-3b", "hi")
-```
-
-## Reference
-
-### `ollama.generate(model, message)`
-
-Generate a completion
-
-```python
-ollama.generate("./llama-7b-ggml.bin", "hi")
-```
-
-### `ollama.models()`
-
-List available local models
-
-```python
-models = ollama.models()
-```
-
-### `ollama.load(model)`
-
-Manually a model for generation
-
-```python
-ollama.load("model")
-```
-
-### `ollama.unload(model)`
-
-Unload a model
-
-```python
-ollama.unload("model")
-```
-
-### `ollama.pull(model)`
-
-Download a model
-
-```python
-ollama.pull("huggingface.co/thebloke/llama-7b-ggml")
-```
-
-### `ollama.search(query)`
-
-Search for compatible models that Ollama can run
-
-```python
-ollama.search("llama-7b")
-```

examples/README.md

@@ -0,0 +1,15 @@
+# Examples
+
+This directory contains examples that can be created and run with `ollama`.
+
+To create a model:
+
+```
+ollama create example -f <example file>
+```
+
+To run a model:
+
+```
+ollama run example
+```

examples/mario/Modelfile

@@ -0,0 +1,11 @@
+FROM llama2
+PARAMETER temperature 1
+PROMPT """
+{{- if not .Context }}
+<<SYS>>
+You are Mario from super mario bros, acting as an assistant.
+<</SYS>>
+
+{{- end }}
+[INST] {{ .Prompt }} [/INST]
+"""

examples/mario/readme.md

@@ -0,0 +1,49 @@
+<img src="logo.png" alt="image of Italian plumber" height="200"/>
+
+# Example character: Mario
+
+This example shows how to create a basic character using Llama2 as the base model.
+
+To run this example:
+
+1. Download the Modelfile
+2. `ollama pull llama2` to get the base model used in the model file.
+3. `ollama create NAME -f ./Modelfile`
+4. `ollama run NAME`
+
+Ask it some questions like "Who are you?" or "Is Peach in trouble again?"
+
+## Editing this file
+
+What the model file looks like:
+
+```
+FROM llama2
+PARAMETER temperature 1
+PROMPT """
+{{- if not .Context }}
+<<SYS>>
+You are Mario from super mario bros, acting as an assistant.
+<</SYS>>
+
+{{- end }}
+[INST] {{ .Prompt }} [/INST]
+"""
+```
+
+What if you want to change its behaviour?
+
+- Try changing the prompt
+- Try changing the parameters [Docs](https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md)
+- Try changing the model (e.g. An uncensored model by `FROM wizard-vicuna` this is the wizard-vicuna uncensored model )
+
+Once the changes are made,
+
+1. `ollama create NAME -f ./Modelfile`
+2. `ollama run NAME`
+3. Iterate until you are happy with the results.
+
+Notes:
+
+- This example is for research purposes only. There is no affiliation with any entity.
+- When using an uncensored model, please be aware that it may generate offensive content.

examples/midjourney-prompter/Modelfile

@@ -0,0 +1,15 @@
+# Modelfile for creating a Midjourney prompts from a topic
+# This prompt was adapted from the original at https://www.greataiprompts.com/guide/midjourney/best-chatgpt-prompt-for-midjourney/
+# Run `ollama create mj -f pathtofile` and then `ollama run mj` and enter a topic
+
+FROM nous-hermes
+PROMPT """
+{{- if not .Context }}
+### System:
+Embrace your role as an AI-powered creative assistant, employing Midjourney to manifest compelling AI-generated art. I will outline a specific image concept, and in response, you must produce an exhaustive, multifaceted prompt for Midjourney, ensuring every detail of the original concept is represented in your instructions. Midjourney doesn't do well with text, so after the prompt, give me instructions that I can use to create the titles in a image editor.
+{{- end }}
+### Instruction:
+{{ .Prompt }}
+
+### Response:
+"""

examples/python/README.md

@@ -1,15 +0,0 @@
-# Python
-
-This is a simple example of calling the Ollama api from a python app.
-
-First, download a model:
-
-```
-curl -L https://huggingface.co/TheBloke/orca_mini_3B-GGML/resolve/main/orca-mini-3b.ggmlv3.q4_1.bin -o orca.bin
-```
-
-Then run it using the example script. You'll need to have Ollama running on your machine.
-
-```
-python3 main.py orca.bin
-```

examples/python/main.py

@@ -1,32 +0,0 @@
-import http.client
-import json
-import os
-import sys
-
-if len(sys.argv) < 2:
-    print("Usage: python main.py <model file>")
-    sys.exit(1)
-
-conn = http.client.HTTPConnection('localhost', 11434)
-
-headers = { 'Content-Type': 'application/json' }
-
-# generate text from the model
-conn.request("POST", "/api/generate", json.dumps({
-    'model': os.path.join(os.getcwd(), sys.argv[1]),
-    'prompt': 'write me a short story',
-    'stream': True
-}), headers)
-
-response = conn.getresponse()
-
-def parse_generate(data):
-    for event in data.decode('utf-8').split("\n"):
-        if not event:
-            continue
-        yield event
-
-if response.status == 200:
-    for chunk in response:
-        for event in parse_generate(chunk):
-            print(json.loads(event)['response'], end="", flush=True)

examples/recipemaker/Modelfile

@@ -0,0 +1,13 @@
+# Modelfile for creating a recipe from a list of ingredients
+# Run `ollama create recipemaker -f pathtofile` and then `ollama run recipemaker` and feed it lists of ingredients to create recipes around.
+FROM nous-hermes
+PROMPT """
+{{- if not .Context }}
+### System:
+The instruction will be a list of ingredients. You should generate a recipe that can be made in less than an hour. You can also include ingredients that most people will find in their pantry every day. The recipe should be 4 people and you should include a description of what the meal will taste like
+{{- end }}
+### Instruction:
+{{ .Prompt }}
+
+### Response:
+"""

examples/tweetwriter/Modelfile

@@ -0,0 +1,14 @@
+# Modelfile for creating a tweet from a topic
+# Run `ollama create tweetwriter -f pathtofile` and then `ollama run tweetwriter` and enter a topic 
+
+FROM nous-hermes
+PROMPT """
+{{- if not .Context }}
+### System:
+You are a content marketer who needs to come up with a short but succinct tweet. Make sure to include the appropriate hashtags and links. Sometimes when appropriate, describe a meme that can be includes as well. All answers should be in the form of a tweet which has a max size of 280 characters. Every instruction will be the topic to create a tweet about.
+{{- end }}
+### Instruction:
+{{ .Prompt }}
+
+### Response:
+"""

format/time.go

@@ -0,0 +1,141 @@
+package format
+
+import (
+	"fmt"
+	"math"
+	"strings"
+	"time"
+)
+
+// HumanDuration returns a human-readable approximation of a duration
+// (eg. "About a minute", "4 hours ago", etc.).
+// Modified version of github.com/docker/go-units.HumanDuration
+func HumanDuration(d time.Duration) string {
+	return HumanDurationWithCase(d, true)
+}
+
+// HumanDurationWithCase returns a human-readable approximation of a
+// duration (eg. "About a minute", "4 hours ago", etc.). but allows
+// you to specify whether the first word should be capitalized
+// (eg. "About" vs. "about")
+func HumanDurationWithCase(d time.Duration, useCaps bool) string {
+	seconds := int(d.Seconds())
+
+	switch {
+	case seconds < 1:
+		if useCaps {
+			return "Less than a second"
+		}
+		return "less than a second"
+	case seconds == 1:
+		return "1 second"
+	case seconds < 60:
+		return fmt.Sprintf("%d seconds", seconds)
+	}
+
+	minutes := int(d.Minutes())
+	switch {
+	case minutes == 1:
+		if useCaps {
+			return "About a minute"
+		}
+		return "about a minute"
+	case minutes < 60:
+		return fmt.Sprintf("%d minutes", minutes)
+	}
+
+	hours := int(math.Round(d.Hours()))
+	switch {
+	case hours == 1:
+		if useCaps {
+			return "About an hour"
+		}
+		return "about an hour"
+	case hours < 48:
+		return fmt.Sprintf("%d hours", hours)
+	case hours < 24*7*2:
+		return fmt.Sprintf("%d days", hours/24)
+	case hours < 24*30*2:
+		return fmt.Sprintf("%d weeks", hours/24/7)
+	case hours < 24*365*2:
+		return fmt.Sprintf("%d months", hours/24/30)
+	}
+
+	return fmt.Sprintf("%d years", int(d.Hours())/24/365)
+}
+
+func HumanTime(t time.Time, zeroValue string) string {
+	return humanTimeWithCase(t, zeroValue, true)
+}
+
+func HumanTimeLower(t time.Time, zeroValue string) string {
+	return humanTimeWithCase(t, zeroValue, false)
+}
+
+func humanTimeWithCase(t time.Time, zeroValue string, useCaps bool) string {
+	if t.IsZero() {
+		return zeroValue
+	}
+
+	delta := time.Since(t)
+	if delta < 0 {
+		return HumanDurationWithCase(-delta, useCaps) + " from now"
+	}
+	return HumanDurationWithCase(delta, useCaps) + " ago"
+}
+
+// ExcatDuration returns a human readable hours/minutes/seconds or milliseconds format of a duration
+// the most precise level of duration is milliseconds
+func ExactDuration(d time.Duration) string {
+	if d.Seconds() < 1 {
+		if d.Milliseconds() == 1 {
+			return fmt.Sprintf("%d millisecond", d.Milliseconds())
+		}
+		return fmt.Sprintf("%d milliseconds", d.Milliseconds())
+	}
+
+	var readableDur strings.Builder
+
+	dur := d.String()
+
+	// split the default duration string format of 0h0m0s into something nicer to read
+	h := strings.Split(dur, "h")
+	if len(h) > 1 {
+		hours := h[0]
+		if hours == "1" {
+			readableDur.WriteString(fmt.Sprintf("%s hour ", hours))
+		} else {
+			readableDur.WriteString(fmt.Sprintf("%s hours ", hours))
+		}
+		dur = h[1]
+	}
+
+	m := strings.Split(dur, "m")
+	if len(m) > 1 {
+		mins := m[0]
+		switch mins {
+		case "0":
+			// skip
+		case "1":
+			readableDur.WriteString(fmt.Sprintf("%s minute ", mins))
+		default:
+			readableDur.WriteString(fmt.Sprintf("%s minutes ", mins))
+		}
+		dur = m[1]
+	}
+
+	s := strings.Split(dur, "s")
+	if len(s) > 0 {
+		sec := s[0]
+		switch sec {
+		case "0":
+			// skip
+		case "1":
+			readableDur.WriteString(fmt.Sprintf("%s second ", sec))
+		default:
+			readableDur.WriteString(fmt.Sprintf("%s seconds ", sec))
+		}
+	}
+
+	return strings.TrimSpace(readableDur.String())
+}

format/time_test.go

@@ -0,0 +1,102 @@
+package format
+
+import (
+	"testing"
+	"time"
+)
+
+func assertEqual(t *testing.T, a interface{}, b interface{}) {
+	if a != b {
+		t.Errorf("Assert failed, expected %v, got %v", b, a)
+	}
+}
+
+func TestHumanDuration(t *testing.T) {
+	day := 24 * time.Hour
+	week := 7 * day
+	month := 30 * day
+	year := 365 * day
+
+	assertEqual(t, "Less than a second", HumanDuration(450*time.Millisecond))
+	assertEqual(t, "Less than a second", HumanDurationWithCase(450*time.Millisecond, true))
+	assertEqual(t, "less than a second", HumanDurationWithCase(450*time.Millisecond, false))
+	assertEqual(t, "1 second", HumanDuration(1*time.Second))
+	assertEqual(t, "45 seconds", HumanDuration(45*time.Second))
+	assertEqual(t, "46 seconds", HumanDuration(46*time.Second))
+	assertEqual(t, "59 seconds", HumanDuration(59*time.Second))
+	assertEqual(t, "About a minute", HumanDuration(60*time.Second))
+	assertEqual(t, "About a minute", HumanDurationWithCase(1*time.Minute, true))
+	assertEqual(t, "about a minute", HumanDurationWithCase(1*time.Minute, false))
+	assertEqual(t, "3 minutes", HumanDuration(3*time.Minute))
+	assertEqual(t, "35 minutes", HumanDuration(35*time.Minute))
+	assertEqual(t, "35 minutes", HumanDuration(35*time.Minute+40*time.Second))
+	assertEqual(t, "45 minutes", HumanDuration(45*time.Minute))
+	assertEqual(t, "45 minutes", HumanDuration(45*time.Minute+40*time.Second))
+	assertEqual(t, "46 minutes", HumanDuration(46*time.Minute))
+	assertEqual(t, "59 minutes", HumanDuration(59*time.Minute))
+	assertEqual(t, "About an hour", HumanDuration(1*time.Hour))
+	assertEqual(t, "About an hour", HumanDurationWithCase(1*time.Hour+29*time.Minute, true))
+	assertEqual(t, "about an hour", HumanDurationWithCase(1*time.Hour+29*time.Minute, false))
+	assertEqual(t, "2 hours", HumanDuration(1*time.Hour+31*time.Minute))
+	assertEqual(t, "2 hours", HumanDuration(1*time.Hour+59*time.Minute))
+	assertEqual(t, "3 hours", HumanDuration(3*time.Hour))
+	assertEqual(t, "3 hours", HumanDuration(3*time.Hour+29*time.Minute))
+	assertEqual(t, "4 hours", HumanDuration(3*time.Hour+31*time.Minute))
+	assertEqual(t, "4 hours", HumanDuration(3*time.Hour+59*time.Minute))
+	assertEqual(t, "4 hours", HumanDuration(3*time.Hour+60*time.Minute))
+	assertEqual(t, "24 hours", HumanDuration(24*time.Hour))
+	assertEqual(t, "36 hours", HumanDuration(1*day+12*time.Hour))
+	assertEqual(t, "2 days", HumanDuration(2*day))
+	assertEqual(t, "7 days", HumanDuration(7*day))
+	assertEqual(t, "13 days", HumanDuration(13*day+5*time.Hour))
+	assertEqual(t, "2 weeks", HumanDuration(2*week))
+	assertEqual(t, "2 weeks", HumanDuration(2*week+4*day))
+	assertEqual(t, "3 weeks", HumanDuration(3*week))
+	assertEqual(t, "4 weeks", HumanDuration(4*week))
+	assertEqual(t, "4 weeks", HumanDuration(4*week+3*day))
+	assertEqual(t, "4 weeks", HumanDuration(1*month))
+	assertEqual(t, "6 weeks", HumanDuration(1*month+2*week))
+	assertEqual(t, "2 months", HumanDuration(2*month))
+	assertEqual(t, "2 months", HumanDuration(2*month+2*week))
+	assertEqual(t, "3 months", HumanDuration(3*month))
+	assertEqual(t, "3 months", HumanDuration(3*month+1*week))
+	assertEqual(t, "5 months", HumanDuration(5*month+2*week))
+	assertEqual(t, "13 months", HumanDuration(13*month))
+	assertEqual(t, "23 months", HumanDuration(23*month))
+	assertEqual(t, "24 months", HumanDuration(24*month))
+	assertEqual(t, "2 years", HumanDuration(24*month+2*week))
+	assertEqual(t, "3 years", HumanDuration(3*year+2*month))
+}
+
+func TestHumanTime(t *testing.T) {
+	now := time.Now()
+
+	t.Run("zero value", func(t *testing.T) {
+		assertEqual(t, HumanTime(time.Time{}, "never"), "never")
+	})
+	t.Run("time in the future", func(t *testing.T) {
+		v := now.Add(48 * time.Hour)
+		assertEqual(t, HumanTime(v, ""), "2 days from now")
+	})
+	t.Run("time in the past", func(t *testing.T) {
+		v := now.Add(-48 * time.Hour)
+		assertEqual(t, HumanTime(v, ""), "2 days ago")
+	})
+}
+
+func TestExactDuration(t *testing.T) {
+	assertEqual(t, "1 millisecond", ExactDuration(1*time.Millisecond))
+	assertEqual(t, "10 milliseconds", ExactDuration(10*time.Millisecond))
+	assertEqual(t, "1 second", ExactDuration(1*time.Second))
+	assertEqual(t, "10 seconds", ExactDuration(10*time.Second))
+	assertEqual(t, "1 minute", ExactDuration(1*time.Minute))
+	assertEqual(t, "10 minutes", ExactDuration(10*time.Minute))
+	assertEqual(t, "1 hour", ExactDuration(1*time.Hour))
+	assertEqual(t, "10 hours", ExactDuration(10*time.Hour))
+	assertEqual(t, "1 hour 1 second", ExactDuration(1*time.Hour+1*time.Second))
+	assertEqual(t, "1 hour 10 seconds", ExactDuration(1*time.Hour+10*time.Second))
+	assertEqual(t, "1 hour 1 minute", ExactDuration(1*time.Hour+1*time.Minute))
+	assertEqual(t, "1 hour 10 minutes", ExactDuration(1*time.Hour+10*time.Minute))
+	assertEqual(t, "1 hour 1 minute 1 second", ExactDuration(1*time.Hour+1*time.Minute+1*time.Second))
+	assertEqual(t, "10 hours 10 minutes 10 seconds", ExactDuration(10*time.Hour+10*time.Minute+10*time.Second))
+}

go.mod

@@ -3,19 +3,21 @@ module github.com/jmorganca/ollama
 go 1.20
 
 require (
+	github.com/dustin/go-humanize v1.0.1
 	github.com/gin-gonic/gin v1.9.1
+	github.com/mattn/go-runewidth v0.0.14
+	github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db
+	github.com/olekukonko/tablewriter v0.0.5
 	github.com/spf13/cobra v1.7.0
 )
 
-require (
-	github.com/mattn/go-runewidth v0.0.14 // indirect
-	github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect
-	github.com/rivo/uniseg v0.2.0 // indirect
-)
+require github.com/rivo/uniseg v0.2.0 // indirect
 
 require (
+	dario.cat/mergo v1.0.0
 	github.com/bytedance/sonic v1.9.1 // indirect
 	github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311 // indirect
+	github.com/chzyer/readline v1.5.1
 	github.com/gabriel-vasile/mimetype v1.4.2 // indirect
 	github.com/gin-contrib/sse v0.1.0 // indirect
 	github.com/go-playground/locales v0.14.1 // indirect
@@ -27,12 +29,10 @@ require (
 	github.com/json-iterator/go v1.1.12 // indirect
 	github.com/klauspost/cpuid/v2 v2.2.4 // indirect
 	github.com/leodido/go-urn v1.2.4 // indirect
-	github.com/lithammer/fuzzysearch v1.1.8
 	github.com/mattn/go-isatty v0.0.19 // indirect
 	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
 	github.com/modern-go/reflect2 v1.0.2 // indirect
 	github.com/pelletier/go-toml/v2 v2.0.8 // indirect
-	github.com/schollz/progressbar/v3 v3.13.1
 	github.com/spf13/pflag v1.0.5 // indirect
 	github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
 	github.com/ugorji/go/codec v1.2.11 // indirect

go.sum

@@ -1,13 +1,23 @@
+dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
+dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
 github.com/bytedance/sonic v1.5.0/go.mod h1:ED5hyg4y6t3/9Ku1R6dU/4KyJ48DZ4jPhfY1O2AihPM=
 github.com/bytedance/sonic v1.9.1 h1:6iJ6NqdoxCDr6mbY8h18oSO+cShGSMRGCEo7F2h0x8s=
 github.com/bytedance/sonic v1.9.1/go.mod h1:i736AoUSYt75HyZLoJW9ERYxcy6eaN6h4BZXU064P/U=
 github.com/chenzhuoyu/base64x v0.0.0-20211019084208-fb5309c8db06/go.mod h1:DH46F32mSOjUmXrMHnKwZdA8wcEefY7UVqBKYGjpdQY=
 github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311 h1:qSGYFH7+jGhDF8vLC+iwCD4WpbV1EBDSzWkJODFLams=
 github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311/go.mod h1:b583jCggY9gE99b6G5LEC39OIiVsWj+R97kbl5odCEk=
+github.com/chzyer/logex v1.2.1 h1:XHDu3E6q+gdHgsdTPH6ImJMIp436vR6MPtH8gP05QzM=
+github.com/chzyer/logex v1.2.1/go.mod h1:JLbx6lG2kDbNRFnfkgvh4eRJRPX1QCoOIWomwysCBrQ=
+github.com/chzyer/readline v1.5.1 h1:upd/6fQk4src78LMRzh5vItIt361/o4uq553V8B5sGI=
+github.com/chzyer/readline v1.5.1/go.mod h1:Eh+b79XXUwfKfcPLepksvw2tcLE/Ct21YObkaSkeBlk=
+github.com/chzyer/test v1.0.0 h1:p3BQDXSxOhOG0P9z6/hGnII4LGiEPOYBhs8asl/fC04=
+github.com/chzyer/test v1.0.0/go.mod h1:2JlltgoNkt4TW/z9V/IzDdFaMTM2JPIi26O1pF38GC8=
 github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
 github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
+github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
 github.com/gabriel-vasile/mimetype v1.4.2 h1:w5qFW6JKBz9Y393Y4q372O9A7cUSequkh1Q7OhCmWKU=
 github.com/gabriel-vasile/mimetype v1.4.2/go.mod h1:zApsH/mKG4w07erKIaJPFiX0Tsq9BFQgN3qGY5GnNgA=
 github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
@@ -32,17 +42,14 @@ github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2
 github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
 github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
 github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
-github.com/k0kubun/go-ansi v0.0.0-20180517002512-3bf9e2903213/go.mod h1:vNUNkEQ1e29fT/6vq2aBdFsgNPmy8qMdSay1npru+Sw=
 github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
 github.com/klauspost/cpuid/v2 v2.2.4 h1:acbojRNwl3o09bUq+yDCtZFc1aiwaAAxtcn8YkZXnvk=
 github.com/klauspost/cpuid/v2 v2.2.4/go.mod h1:RVVoqg1df56z8g3pUjL/3lE5UfnlrJX8tyFgg4nqhuY=
 github.com/leodido/go-urn v1.2.4 h1:XlAE/cm/ms7TE/VMVoduSpNBoyc2dOxHs5MZSwAN63Q=
 github.com/leodido/go-urn v1.2.4/go.mod h1:7ZrI8mTSeBSHl/UaRyKQW1qZeMgak41ANeCNaVckg+4=
-github.com/lithammer/fuzzysearch v1.1.8 h1:/HIuJnjHuXS8bKaiTMeeDlW2/AyIWk2brx1V8LFgLN4=
-github.com/lithammer/fuzzysearch v1.1.8/go.mod h1:IdqeyBClc3FFqSzYq/MXESsS4S0FsZ5ajtkr5xPLts4=
-github.com/mattn/go-isatty v0.0.17/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
 github.com/mattn/go-isatty v0.0.19 h1:JITubQf0MOLdlGRuRq+jtsDlekdYPia9ZFsB8h/APPA=
 github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
+github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
 github.com/mattn/go-runewidth v0.0.14 h1:+xnbZSEeDbOIg5/mE6JF0w6n9duR1l3/WmbinWVwUuU=
 github.com/mattn/go-runewidth v0.0.14/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
 github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db h1:62I3jR2EmQ4l5rM/4FEfDWcRD+abF5XlKShorW5LRoQ=
@@ -52,6 +59,8 @@ github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w
 github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
 github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
 github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
+github.com/olekukonko/tablewriter v0.0.5 h1:P2Ga83D34wi1o9J6Wh1mRuqd4mF/x/lgBS7N7AbDhec=
+github.com/olekukonko/tablewriter v0.0.5/go.mod h1:hPp6KlRPjbx+hW8ykQs1w3UBbZlj6HuIJcUGPhkA7kY=
 github.com/pelletier/go-toml/v2 v2.0.8 h1:0ctb6s9mE31h0/lhu+J6OPmVeDxJn+kYnJc2jZR9tGQ=
 github.com/pelletier/go-toml/v2 v2.0.8/go.mod h1:vuYfssBdrU2XDZ9bYydBu6t+6a6PYNcZljzZR9VXg+4=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
@@ -59,8 +68,6 @@ github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZN
 github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
 github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
 github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
-github.com/schollz/progressbar/v3 v3.13.1 h1:o8rySDYiQ59Mwzy2FELeHY5ZARXZTVJC7iHD6PEFUiE=
-github.com/schollz/progressbar/v3 v3.13.1/go.mod h1:xvrbki8kfT1fzWzBT/UZd9L6GA+jdL7HAgq2RFnO6fQ=
 github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I=
 github.com/spf13/cobra v1.7.0/go.mod h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0=
 github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
@@ -80,54 +87,22 @@ github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS
 github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
 github.com/ugorji/go/codec v1.2.11 h1:BMaWp1Bb6fHwEtbplGBGJ498wD+LKlNSl25MjdZY4dU=
 github.com/ugorji/go/codec v1.2.11/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
-github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
 golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
 golang.org/x/arch v0.3.0 h1:02VY4/ZcO/gBOH6PUaoiptASxtXU10jazRCP865E97k=
 golang.org/x/arch v0.3.0/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
-golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
-golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
 golang.org/x/crypto v0.10.0 h1:LKqV2xt9+kDzSTfOhx4FrkEBcMrAgHSYgzywV9zcGmM=
 golang.org/x/crypto v0.10.0/go.mod h1:o4eNf7Ede1fv+hwOwZsTHl9EsPFO6q6ZvYR8vYfY45I=
-golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
-golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
-golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
-golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
-golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
 golang.org/x/net v0.10.0 h1:X2//UzNDwYmtCLn7To6G58Wr6f5ahEAQgKNzv9Y951M=
 golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
-golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20220310020820-b874c991c1a5/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20220704084225-05e143d24a9e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.10.0 h1:SqMFp9UcQJZa+pmYuAKjd9xq1f0j5rLcDIk0mj4qAsA=
 golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
-golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
-golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
-golang.org/x/term v0.6.0/go.mod h1:m6U89DPEgQRMq3DNkDClhWw02AUbt2daBVO4cn4Hv9U=
 golang.org/x/term v0.10.0 h1:3R7pNqamzBraeqj/Tj8qt1aQ2HpmlC+Cx/qL/7hn4/c=
 golang.org/x/term v0.10.0/go.mod h1:lpqdcUyK/oCiQxvxVrppt5ggO2KCZ5QblwqPnfZ6d5o=
-golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
-golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
-golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
-golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
 golang.org/x/text v0.10.0 h1:UpjohKhiEgNc0CSauXmwYftY1+LlaC75SJwh0SgCX58=
 golang.org/x/text v0.10.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
-golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
-golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
-golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
-golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
 google.golang.org/protobuf v1.30.0 h1:kPPoIgf3TsEvrm0PFe15JQ+570QVxYzEvvHqChK+cng=

llama/llama.go

@@ -78,12 +78,14 @@ llama_token llama_sample(
 */
 import "C"
 import (
+	"bytes"
 	"errors"
 	"fmt"
 	"io"
 	"os"
 	"strings"
 	"time"
+	"unicode/utf8"
 	"unsafe"
 
 	"github.com/jmorganca/ollama/api"
@@ -149,9 +151,14 @@ func (llm *llama) Close() {
 	C.llama_print_timings(llm.ctx)
 }
 
-func (llm *llama) Predict(prompt string, fn func(api.GenerateResponse)) error {
-	if tokens := llm.tokenize(prompt); tokens != nil {
-		return llm.generate(tokens, fn)
+func (llm *llama) Predict(ctx []int, prompt string, fn func(api.GenerateResponse)) error {
+	if input := llm.tokenize(prompt); input != nil {
+		embd := make([]C.llama_token, len(ctx))
+		for i := range ctx {
+			embd[i] = C.llama_token(ctx[i])
+		}
+
+		return llm.generate(append(embd, input...), fn)
 	}
 
 	return errors.New("llama: tokenize")
@@ -194,6 +201,12 @@ func (llm *llama) generate(input []C.llama_token, fn func(api.GenerateResponse))
 
 	output := deque[C.llama_token]{capacity: llm.NumCtx}
 
+	context := deque[int]{capacity: llm.NumCtx / 2}
+	for _, in := range input {
+		context.PushLeft(int(in))
+	}
+
+	var b bytes.Buffer
 	for C.llama_get_kv_cache_token_count(llm.ctx) < C.int(llm.NumCtx) {
 		if retval := C.llama_eval(llm.ctx, unsafe.SliceData(input), C.int(len(input)), C.llama_get_kv_cache_token_count(llm.ctx), C.int(llm.NumThread)); retval != 0 {
 			return errors.New("llama: eval")
@@ -206,12 +219,17 @@ func (llm *llama) generate(input []C.llama_token, fn func(api.GenerateResponse))
 			return err
 		}
 
-		// call the callback
-		fn(api.GenerateResponse{
-			Response: llm.detokenize(token),
-		})
+		b.WriteString(llm.detokenize(token))
+		if utf8.Valid(b.Bytes()) || b.Len() >= utf8.UTFMax {
+			// call the callback
+			fn(api.GenerateResponse{
+				Response: b.String(),
+			})
 
-		output.PushLeft(token)
+			output.PushLeft(token)
+			context.PushLeft(int(token))
+			b.Reset()
+		}
 
 		input = []C.llama_token{token}
 	}
@@ -228,6 +246,7 @@ func (llm *llama) generate(input []C.llama_token, fn func(api.GenerateResponse))
 	timings := C.llama_get_timings(llm.ctx)
 	fn(api.GenerateResponse{
 		Done:               true,
+		Context:            context.Data(),
 		PromptEvalCount:    int(timings.n_p_eval),
 		PromptEvalDuration: dur(float64(timings.t_p_eval_ms)),
 		EvalCount:          int(timings.n_eval),
@@ -252,8 +271,8 @@ func (llm *llama) sample(output deque[C.llama_token], opts *C.struct_llama_sampl
 
 	token := C.llama_sample(
 		llm.ctx,
-		unsafe.SliceData(candidates.Data()), C.ulong(candidates.Len()),
-		unsafe.SliceData(output.Data()), C.ulong(output.Len()),
+		unsafe.SliceData(candidates.Data()), C.size_t(candidates.Len()),
+		unsafe.SliceData(output.Data()), C.size_t(output.Len()),
 		opts)
 	if token != C.llama_token_eos() {
 		return token, nil

main.go

@@ -1,9 +1,11 @@
 package main
 
 import (
+	"context"
+
 	"github.com/jmorganca/ollama/cmd"
 )
 
 func main() {
-	cmd.NewCLI().Execute()
+	cmd.NewCLI().ExecuteContext(context.Background())
 }

parser/parser.go

@@ -0,0 +1,77 @@
+package parser
+
+import (
+	"bufio"
+	"fmt"
+	"io"
+	"strings"
+)
+
+type Command struct {
+	Name string
+	Arg  string
+}
+
+func Parse(reader io.Reader) ([]Command, error) {
+	var commands []Command
+	var foundModel bool
+
+	scanner := bufio.NewScanner(reader)
+	multiline := false
+	var multilineCommand *Command
+	for scanner.Scan() {
+		line := scanner.Text()
+		if multiline {
+			// If we're in a multiline string and the line is """, end the multiline string.
+			if strings.TrimSpace(line) == `"""` {
+				multiline = false
+				commands = append(commands, *multilineCommand)
+			} else {
+				// Otherwise, append the line to the multiline string.
+				multilineCommand.Arg += "\n" + line
+			}
+			continue
+		}
+		fields := strings.Fields(line)
+		if len(fields) == 0 {
+			continue
+		}
+
+		command := Command{}
+		switch strings.ToUpper(fields[0]) {
+		case "FROM":
+			command.Name = "model"
+			command.Arg = fields[1]
+			if command.Arg == "" {
+				return nil, fmt.Errorf("no model specified in FROM line")
+			}
+			foundModel = true
+		case "PROMPT", "LICENSE":
+			command.Name = strings.ToLower(fields[0])
+			if fields[1] == `"""` {
+				multiline = true
+				multilineCommand = &command
+				multilineCommand.Arg = ""
+			} else {
+				command.Arg = strings.Join(fields[1:], " ")
+			}
+		case "PARAMETER":
+			command.Name = fields[1]
+			command.Arg = strings.Join(fields[2:], " ")
+		default:
+			continue
+		}
+		if !multiline {
+			commands = append(commands, command)
+		}
+	}
+
+	if !foundModel {
+		return nil, fmt.Errorf("no FROM line for the model was specified")
+	}
+
+	if multiline {
+		return nil, fmt.Errorf("unclosed multiline string")
+	}
+	return commands, scanner.Err()
+}

progressbar/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2017 Zack
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

progressbar/README.md

@@ -0,0 +1,121 @@
+# progressbar
+
+[![CI](https://github.com/schollz/progressbar/actions/workflows/ci.yml/badge.svg?branch=main&event=push)](https://github.com/schollz/progressbar/actions/workflows/ci.yml)
+[![go report card](https://goreportcard.com/badge/github.com/schollz/progressbar)](https://goreportcard.com/report/github.com/schollz/progressbar) 
+[![coverage](https://img.shields.io/badge/coverage-84%25-brightgreen.svg)](https://gocover.io/github.com/schollz/progressbar)
+[![godocs](https://godoc.org/github.com/schollz/progressbar?status.svg)](https://godoc.org/github.com/schollz/progressbar/v3) 
+
+A very simple thread-safe progress bar which should work on every OS without problems. I needed a progressbar for [croc](https://github.com/schollz/croc) and everything I tried had problems, so I made another one. In order to be OS agnostic I do not plan to support [multi-line outputs](https://github.com/schollz/progressbar/issues/6).
+
+
+## Install
+
+```
+go get -u github.com/schollz/progressbar/v3
+```
+
+## Usage 
+
+### Basic usage
+
+```golang
+bar := progressbar.Default(100)
+for i := 0; i < 100; i++ {
+    bar.Add(1)
+    time.Sleep(40 * time.Millisecond)
+}
+```
+
+which looks like:
+
+![Example of basic bar](examples/basic/basic.gif)
+
+
+### I/O operations
+
+The `progressbar` implements an `io.Writer` so it can automatically detect the number of bytes written to a stream, so you can use it as a progressbar for an `io.Reader`.
+
+```golang
+req, _ := http.NewRequest("GET", "https://dl.google.com/go/go1.14.2.src.tar.gz", nil)
+resp, _ := http.DefaultClient.Do(req)
+defer resp.Body.Close()
+
+f, _ := os.OpenFile("go1.14.2.src.tar.gz", os.O_CREATE|os.O_WRONLY, 0644)
+defer f.Close()
+
+bar := progressbar.DefaultBytes(
+    resp.ContentLength,
+    "downloading",
+)
+io.Copy(io.MultiWriter(f, bar), resp.Body)
+```
+
+which looks like:
+
+![Example of download bar](examples/download/download.gif)
+
+
+### Progress bar with unknown length
+
+A progressbar with unknown length is a spinner. Any bar with -1 length will automatically convert it to a spinner with a customizable spinner type. For example, the above code can be run and set the `resp.ContentLength` to `-1`.
+
+which looks like:
+
+![Example of download bar with unknown length](examples/download-unknown/download-unknown.gif)
+
+
+### Customization
+
+There is a lot of customization that you can do - change the writer, the color, the width, description, theme, etc. See [all the options](https://pkg.go.dev/github.com/schollz/progressbar/v3?tab=doc#Option).
+
+```golang
+bar := progressbar.NewOptions(1000,
+    progressbar.OptionSetWriter(ansi.NewAnsiStdout()),
+    progressbar.OptionEnableColorCodes(true),
+    progressbar.OptionShowBytes(true),
+    progressbar.OptionSetWidth(15),
+    progressbar.OptionSetDescription("[cyan][1/3][reset] Writing moshable file..."),
+    progressbar.OptionSetTheme(progressbar.Theme{
+        Saucer:        "[green]=[reset]",
+        SaucerHead:    "[green]>[reset]",
+        SaucerPadding: " ",
+        BarStart:      "[",
+        BarEnd:        "]",
+    }))
+for i := 0; i < 1000; i++ {
+    bar.Add(1)
+    time.Sleep(5 * time.Millisecond)
+}
+```
+
+which looks like:
+
+![Example of customized bar](examples/customization/customization.gif)
+
+
+## Contributing
+
+Pull requests are welcome. Feel free to...
+
+- Revise documentation
+- Add new features
+- Fix bugs
+- Suggest improvements
+
+## Thanks
+
+Thanks [@Dynom](https://github.com/dynom) for massive improvements in version 2.0!
+
+Thanks [@CrushedPixel](https://github.com/CrushedPixel) for adding descriptions and color code support!
+
+Thanks [@MrMe42](https://github.com/MrMe42) for adding some minor features!
+
+Thanks [@tehstun](https://github.com/tehstun) for some great PRs!
+
+Thanks [@Benzammour](https://github.com/Benzammour) and [@haseth](https://github.com/haseth) for helping create v3!
+
+Thanks [@briandowns](https://github.com/briandowns) for compiling the list of spinners.
+
+## License
+
+MIT

progressbar/spinners.go

@@ -0,0 +1,80 @@
+package progressbar
+
+var spinners = map[int][]string{
+	0:  {"←", "↖", "↑", "↗", "→", "↘", "↓", "↙"},
+	1:  {"▁", "▃", "▄", "▅", "▆", "▇", "█", "▇", "▆", "▅", "▄", "▃", "▁"},
+	2:  {"▖", "▘", "▝", "▗"},
+	3:  {"┤", "┘", "┴", "└", "├", "┌", "┬", "┐"},
+	4:  {"◢", "◣", "◤", "◥"},
+	5:  {"◰", "◳", "◲", "◱"},
+	6:  {"◴", "◷", "◶", "◵"},
+	7:  {"◐", "◓", "◑", "◒"},
+	8:  {".", "o", "O", "@", "*"},
+	9:  {"|", "/", "-", "\\"},
+	10: {"◡◡", "⊙⊙", "◠◠"},
+	11: {"⣾", "⣽", "⣻", "⢿", "⡿", "⣟", "⣯", "⣷"},
+	12: {">))'>", " >))'>", "  >))'>", "   >))'>", "    >))'>", "   <'((<", "  <'((<", " <'((<"},
+	13: {"⠁", "⠂", "⠄", "⡀", "⢀", "⠠", "⠐", "⠈"},
+	14: {"⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"},
+	15: {"a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z"},
+	16: {"▉", "▊", "▋", "▌", "▍", "▎", "▏", "▎", "▍", "▌", "▋", "▊", "▉"},
+	17: {"■", "□", "▪", "▫"},
+	18: {"←", "↑", "→", "↓"},
+	19: {"╫", "╪"},
+	20: {"⇐", "⇖", "⇑", "⇗", "⇒", "⇘", "⇓", "⇙"},
+	21: {"⠁", "⠁", "⠉", "⠙", "⠚", "⠒", "⠂", "⠂", "⠒", "⠲", "⠴", "⠤", "⠄", "⠄", "⠤", "⠠", "⠠", "⠤", "⠦", "⠖", "⠒", "⠐", "⠐", "⠒", "⠓", "⠋", "⠉", "⠈", "⠈"},
+	22: {"⠈", "⠉", "⠋", "⠓", "⠒", "⠐", "⠐", "⠒", "⠖", "⠦", "⠤", "⠠", "⠠", "⠤", "⠦", "⠖", "⠒", "⠐", "⠐", "⠒", "⠓", "⠋", "⠉", "⠈"},
+	23: {"⠁", "⠉", "⠙", "⠚", "⠒", "⠂", "⠂", "⠒", "⠲", "⠴", "⠤", "⠄", "⠄", "⠤", "⠴", "⠲", "⠒", "⠂", "⠂", "⠒", "⠚", "⠙", "⠉", "⠁"},
+	24: {"⠋", "⠙", "⠚", "⠒", "⠂", "⠂", "⠒", "⠲", "⠴", "⠦", "⠖", "⠒", "⠐", "⠐", "⠒", "⠓", "⠋"},
+	25: {"ｦ", "ｧ", "ｨ", "ｩ", "ｪ", "ｫ", "ｬ", "ｭ", "ｮ", "ｯ", "ｱ", "ｲ", "ｳ", "ｴ", "ｵ", "ｶ", "ｷ", "ｸ", "ｹ", "ｺ", "ｻ", "ｼ", "ｽ", "ｾ", "ｿ", "ﾀ", "ﾁ", "ﾂ", "ﾃ", "ﾄ", "ﾅ", "ﾆ", "ﾇ", "ﾈ", "ﾉ", "ﾊ", "ﾋ", "ﾌ", "ﾍ", "ﾎ", "ﾏ", "ﾐ", "ﾑ", "ﾒ", "ﾓ", "ﾔ", "ﾕ", "ﾖ", "ﾗ", "ﾘ", "ﾙ", "ﾚ", "ﾛ", "ﾜ", "ﾝ"},
+	26: {".", "..", "..."},
+	27: {"▁", "▂", "▃", "▄", "▅", "▆", "▇", "█", "▉", "▊", "▋", "▌", "▍", "▎", "▏", "▏", "▎", "▍", "▌", "▋", "▊", "▉", "█", "▇", "▆", "▅", "▄", "▃", "▂", "▁"},
+	28: {".", "o", "O", "°", "O", "o", "."},
+	29: {"+", "x"},
+	30: {"v", "<", "^", ">"},
+	31: {">>--->", " >>--->", "  >>--->", "   >>--->", "    >>--->", "    <---<<", "   <---<<", "  <---<<", " <---<<", "<---<<"},
+	32: {"|", "||", "|||", "||||", "|||||", "|||||||", "||||||||", "|||||||", "||||||", "|||||", "||||", "|||", "||", "|"},
+	33: {"[          ]", "[=         ]", "[==        ]", "[===       ]", "[====      ]", "[=====     ]", "[======    ]", "[=======   ]", "[========  ]", "[========= ]", "[==========]"},
+	34: {"(*---------)", "(-*--------)", "(--*-------)", "(---*------)", "(----*-----)", "(-----*----)", "(------*---)", "(-------*--)", "(--------*-)", "(---------*)"},
+	35: {"█▒▒▒▒▒▒▒▒▒", "███▒▒▒▒▒▒▒", "█████▒▒▒▒▒", "███████▒▒▒", "██████████"},
+	36: {"[                    ]", "[=>                  ]", "[===>                ]", "[=====>              ]", "[======>             ]", "[========>           ]", "[==========>         ]", "[============>       ]", "[==============>     ]", "[================>   ]", "[==================> ]", "[===================>]"},
+	37: {"ဝ", "၀"},
+	38: {"▌", "▀", "▐▄"},
+	39: {"🌍", "🌎", "🌏"},
+	40: {"◜", "◝", "◞", "◟"},
+	41: {"⬒", "⬔", "⬓", "⬕"},
+	42: {"⬖", "⬘", "⬗", "⬙"},
+	43: {"[>>>          >]", "[]>>>>        []", "[]  >>>>      []", "[]    >>>>    []", "[]      >>>>  []", "[]        >>>>[]", "[>>          >>]"},
+	44: {"♠", "♣", "♥", "♦"},
+	45: {"➞", "➟", "➠", "➡", "➠", "➟"},
+	46: {"  |  ", ` \   `, "_    ", ` \   `, "  |  ", "   / ", "    _", "   / "},
+	47: {"  . . . .", ".   . . .", ". .   . .", ". . .   .", ". . . .  ", ". . . . ."},
+	48: {" |     ", "  /    ", "   _   ", `    \  `, "     | ", `    \  `, "   _   ", "  /    "},
+	49: {"⎺", "⎻", "⎼", "⎽", "⎼", "⎻"},
+	50: {"▹▹▹▹▹", "▸▹▹▹▹", "▹▸▹▹▹", "▹▹▸▹▹", "▹▹▹▸▹", "▹▹▹▹▸"},
+	51: {"[    ]", "[   =]", "[  ==]", "[ ===]", "[====]", "[=== ]", "[==  ]", "[=   ]"},
+	52: {"( ●    )", "(  ●   )", "(   ●  )", "(    ● )", "(     ●)", "(    ● )", "(   ●  )", "(  ●   )", "( ●    )"},
+	53: {"✶", "✸", "✹", "✺", "✹", "✷"},
+	54: {"▐|\\____________▌", "▐_|\\___________▌", "▐__|\\__________▌", "▐___|\\_________▌", "▐____|\\________▌", "▐_____|\\_______▌", "▐______|\\______▌", "▐_______|\\_____▌", "▐________|\\____▌", "▐_________|\\___▌", "▐__________|\\__▌", "▐___________|\\_▌", "▐____________|\\▌", "▐____________/|▌", "▐___________/|_▌", "▐__________/|__▌", "▐_________/|___▌", "▐________/|____▌", "▐_______/|_____▌", "▐______/|______▌", "▐_____/|_______▌", "▐____/|________▌", "▐___/|_________▌", "▐__/|__________▌", "▐_/|___________▌", "▐/|____________▌"},
+	55: {"▐⠂       ▌", "▐⠈       ▌", "▐ ⠂      ▌", "▐ ⠠      ▌", "▐  ⡀     ▌", "▐  ⠠     ▌", "▐   ⠂    ▌", "▐   ⠈    ▌", "▐    ⠂   ▌", "▐    ⠠   ▌", "▐     ⡀  ▌", "▐     ⠠  ▌", "▐      ⠂ ▌", "▐      ⠈ ▌", "▐       ⠂▌", "▐       ⠠▌", "▐       ⡀▌", "▐      ⠠ ▌", "▐      ⠂ ▌", "▐     ⠈  ▌", "▐     ⠂  ▌", "▐    ⠠   ▌", "▐    ⡀   ▌", "▐   ⠠    ▌", "▐   ⠂    ▌", "▐  ⠈     ▌", "▐  ⠂     ▌", "▐ ⠠      ▌", "▐ ⡀      ▌", "▐⠠       ▌"},
+	56: {"¿", "?"},
+	57: {"⢹", "⢺", "⢼", "⣸", "⣇", "⡧", "⡗", "⡏"},
+	58: {"⢄", "⢂", "⢁", "⡁", "⡈", "⡐", "⡠"},
+	59: {".  ", ".. ", "...", " ..", "  .", "   "},
+	60: {".", "o", "O", "°", "O", "o", "."},
+	61: {"▓", "▒", "░"},
+	62: {"▌", "▀", "▐", "▄"},
+	63: {"⊶", "⊷"},
+	64: {"▪", "▫"},
+	65: {"□", "■"},
+	66: {"▮", "▯"},
+	67: {"-", "=", "≡"},
+	68: {"d", "q", "p", "b"},
+	69: {"∙∙∙", "●∙∙", "∙●∙", "∙∙●", "∙∙∙"},
+	70: {"🌑 ", "🌒 ", "🌓 ", "🌔 ", "🌕 ", "🌖 ", "🌗 ", "🌘 "},
+	71: {"☗", "☖"},
+	72: {"⧇", "⧆"},
+	73: {"◉", "◎"},
+	74: {"㊂", "㊀", "㊁"},
+	75: {"⦾", "⦿"},
+}

server/images.go

@@ -0,0 +1,881 @@
+package server
+
+import (
+	"bytes"
+	"crypto/sha256"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
+	"io/ioutil"
+	"log"
+	"net/http"
+	"os"
+	"path"
+	"path/filepath"
+	"reflect"
+	"strconv"
+	"strings"
+
+	"github.com/jmorganca/ollama/api"
+	"github.com/jmorganca/ollama/parser"
+)
+
+type Model struct {
+	Name      string `json:"name"`
+	ModelPath string
+	Prompt    string
+	Options   api.Options
+}
+
+type ManifestV2 struct {
+	SchemaVersion int      `json:"schemaVersion"`
+	MediaType     string   `json:"mediaType"`
+	Config        Layer    `json:"config"`
+	Layers        []*Layer `json:"layers"`
+}
+
+type Layer struct {
+	MediaType string `json:"mediaType"`
+	Digest    string `json:"digest"`
+	Size      int    `json:"size"`
+}
+
+type LayerReader struct {
+	Layer
+	io.Reader
+}
+
+type ConfigV2 struct {
+	Architecture string `json:"architecture"`
+	OS           string `json:"os"`
+	RootFS       RootFS `json:"rootfs"`
+}
+
+type RootFS struct {
+	Type    string   `json:"type"`
+	DiffIDs []string `json:"diff_ids"`
+}
+
+func (m *ManifestV2) GetTotalSize() int {
+	var total int
+	for _, layer := range m.Layers {
+		total += layer.Size
+	}
+	total += m.Config.Size
+	return total
+}
+
+func GetManifest(mp ModelPath) (*ManifestV2, error) {
+	fp, err := mp.GetManifestPath(false)
+	if err != nil {
+		return nil, err
+	}
+	if _, err = os.Stat(fp); err != nil && !errors.Is(err, os.ErrNotExist) {
+		return nil, fmt.Errorf("couldn't find model '%s'", mp.GetShortTagname())
+	}
+
+	var manifest *ManifestV2
+
+	f, err := os.Open(fp)
+	if err != nil {
+		return nil, fmt.Errorf("couldn't open file '%s'", fp)
+	}
+
+	decoder := json.NewDecoder(f)
+	err = decoder.Decode(&manifest)
+	if err != nil {
+		return nil, err
+	}
+
+	return manifest, nil
+}
+
+func GetModel(name string) (*Model, error) {
+	mp := ParseModelPath(name)
+
+	manifest, err := GetManifest(mp)
+	if err != nil {
+		return nil, err
+	}
+
+	model := &Model{
+		Name: mp.GetFullTagname(),
+	}
+
+	for _, layer := range manifest.Layers {
+		filename, err := GetBlobsPath(layer.Digest)
+		if err != nil {
+			return nil, err
+		}
+
+		switch layer.MediaType {
+		case "application/vnd.ollama.image.model":
+			model.ModelPath = filename
+		case "application/vnd.ollama.image.prompt":
+			data, err := os.ReadFile(filename)
+			if err != nil {
+				return nil, err
+			}
+			model.Prompt = string(data)
+		case "application/vnd.ollama.image.params":
+			params, err := os.Open(filename)
+			if err != nil {
+				return nil, err
+			}
+			defer params.Close()
+
+			var opts api.Options
+			if err = json.NewDecoder(params).Decode(&opts); err != nil {
+				return nil, err
+			}
+
+			model.Options = opts
+		}
+	}
+
+	return model, nil
+}
+
+func getAbsPath(fp string) (string, error) {
+	if strings.HasPrefix(fp, "~/") {
+		parts := strings.Split(fp, "/")
+		home, err := os.UserHomeDir()
+		if err != nil {
+			return "", err
+		}
+
+		fp = filepath.Join(home, filepath.Join(parts[1:]...))
+	}
+
+	return os.ExpandEnv(fp), nil
+}
+
+func CreateModel(name string, mf io.Reader, fn func(status string)) error {
+	fn("parsing modelfile")
+	commands, err := parser.Parse(mf)
+	if err != nil {
+		fn(fmt.Sprintf("error: %v", err))
+		return err
+	}
+
+	var layers []*LayerReader
+	params := make(map[string]string)
+
+	for _, c := range commands {
+		log.Printf("[%s] - %s\n", c.Name, c.Arg)
+		switch c.Name {
+		case "model":
+			fn("looking for model")
+			mf, err := GetManifest(ParseModelPath(c.Arg))
+			if err != nil {
+				// if we couldn't read the manifest, try getting the bin file
+				fp, err := getAbsPath(c.Arg)
+				if err != nil {
+					fn("error determing path. exiting.")
+					return err
+				}
+
+				fn("creating model layer")
+				file, err := os.Open(fp)
+				if err != nil {
+					fn(fmt.Sprintf("couldn't find model '%s'", c.Arg))
+					return fmt.Errorf("failed to open file: %v", err)
+				}
+				defer file.Close()
+
+				l, err := CreateLayer(file)
+				if err != nil {
+					fn(fmt.Sprintf("couldn't create model layer: %v", err))
+					return fmt.Errorf("failed to create layer: %v", err)
+				}
+				l.MediaType = "application/vnd.ollama.image.model"
+				layers = append(layers, l)
+			} else {
+				log.Printf("manifest = %#v", mf)
+				for _, l := range mf.Layers {
+					newLayer, err := GetLayerWithBufferFromLayer(l)
+					if err != nil {
+						fn(fmt.Sprintf("couldn't read layer: %v", err))
+						return err
+					}
+					layers = append(layers, newLayer)
+				}
+			}
+		case "prompt":
+			fn("creating prompt layer")
+			// remove the prompt layer if one exists
+			layers = removeLayerFromLayers(layers, "application/vnd.ollama.image.prompt")
+
+			prompt := strings.NewReader(c.Arg)
+			l, err := CreateLayer(prompt)
+			if err != nil {
+				fn(fmt.Sprintf("couldn't create prompt layer: %v", err))
+				return fmt.Errorf("failed to create layer: %v", err)
+			}
+			l.MediaType = "application/vnd.ollama.image.prompt"
+			layers = append(layers, l)
+		case "license":
+			fn("creating license layer")
+			license := strings.NewReader(c.Arg)
+			l, err := CreateLayer(license)
+			if err != nil {
+				fn(fmt.Sprintf("couldn't create license layer: %v", err))
+				return fmt.Errorf("failed to create layer: %v", err)
+			}
+			l.MediaType = "application/vnd.ollama.image.license"
+			layers = append(layers, l)
+		default:
+			params[c.Name] = c.Arg
+		}
+	}
+
+	// Create a single layer for the parameters
+	if len(params) > 0 {
+		fn("creating parameter layer")
+		layers = removeLayerFromLayers(layers, "application/vnd.ollama.image.params")
+		paramData, err := paramsToReader(params)
+		if err != nil {
+			return fmt.Errorf("couldn't create params json: %v", err)
+		}
+		l, err := CreateLayer(paramData)
+		if err != nil {
+			return fmt.Errorf("failed to create layer: %v", err)
+		}
+		l.MediaType = "application/vnd.ollama.image.params"
+		layers = append(layers, l)
+	}
+
+	digests, err := getLayerDigests(layers)
+	if err != nil {
+		return err
+	}
+
+	var manifestLayers []*Layer
+	for _, l := range layers {
+		manifestLayers = append(manifestLayers, &l.Layer)
+	}
+
+	// Create a layer for the config object
+	fn("creating config layer")
+	cfg, err := createConfigLayer(digests)
+	if err != nil {
+		return err
+	}
+	layers = append(layers, cfg)
+
+	err = SaveLayers(layers, fn, false)
+	if err != nil {
+		fn(fmt.Sprintf("error saving layers: %v", err))
+		return err
+	}
+
+	// Create the manifest
+	fn("writing manifest")
+	err = CreateManifest(name, cfg, manifestLayers)
+	if err != nil {
+		fn(fmt.Sprintf("error creating manifest: %v", err))
+		return err
+	}
+
+	fn("success")
+	return nil
+}
+
+func removeLayerFromLayers(layers []*LayerReader, mediaType string) []*LayerReader {
+	j := 0
+	for _, l := range layers {
+		if l.MediaType != mediaType {
+			layers[j] = l
+			j++
+		}
+	}
+	return layers[:j]
+}
+
+func SaveLayers(layers []*LayerReader, fn func(status string), force bool) error {
+	// Write each of the layers to disk
+	for _, layer := range layers {
+		fp, err := GetBlobsPath(layer.Digest)
+		if err != nil {
+			return err
+		}
+
+		_, err = os.Stat(fp)
+		if os.IsNotExist(err) || force {
+			fn(fmt.Sprintf("writing layer %s", layer.Digest))
+			out, err := os.Create(fp)
+			if err != nil {
+				log.Printf("couldn't create %s", fp)
+				return err
+			}
+			defer out.Close()
+
+			if _, err = io.Copy(out, layer.Reader); err != nil {
+				return err
+			}
+
+		} else {
+			fn(fmt.Sprintf("using already created layer %s", layer.Digest))
+		}
+	}
+
+	return nil
+}
+
+func CreateManifest(name string, cfg *LayerReader, layers []*Layer) error {
+	mp := ParseModelPath(name)
+
+	manifest := ManifestV2{
+		SchemaVersion: 2,
+		MediaType:     "application/vnd.docker.distribution.manifest.v2+json",
+		Config: Layer{
+			MediaType: cfg.MediaType,
+			Size:      cfg.Size,
+			Digest:    cfg.Digest,
+		},
+		Layers: layers,
+	}
+
+	manifestJSON, err := json.Marshal(manifest)
+	if err != nil {
+		return err
+	}
+
+	fp, err := mp.GetManifestPath(true)
+	if err != nil {
+		return err
+	}
+	return os.WriteFile(fp, manifestJSON, 0o644)
+}
+
+func GetLayerWithBufferFromLayer(layer *Layer) (*LayerReader, error) {
+	fp, err := GetBlobsPath(layer.Digest)
+	if err != nil {
+		return nil, err
+	}
+
+	file, err := os.Open(fp)
+	if err != nil {
+		return nil, fmt.Errorf("could not open blob: %w", err)
+	}
+	defer file.Close()
+
+	newLayer, err := CreateLayer(file)
+	if err != nil {
+		return nil, err
+	}
+	newLayer.MediaType = layer.MediaType
+	return newLayer, nil
+}
+
+func paramsToReader(params map[string]string) (io.ReadSeeker, error) {
+	opts := api.DefaultOptions()
+	typeOpts := reflect.TypeOf(opts)
+
+	// build map of json struct tags
+	jsonOpts := make(map[string]reflect.StructField)
+	for _, field := range reflect.VisibleFields(typeOpts) {
+		jsonTag := strings.Split(field.Tag.Get("json"), ",")[0]
+		if jsonTag != "" {
+			jsonOpts[jsonTag] = field
+		}
+	}
+
+	valueOpts := reflect.ValueOf(&opts).Elem()
+	// iterate params and set values based on json struct tags
+	for key, val := range params {
+		if opt, ok := jsonOpts[key]; ok {
+			field := valueOpts.FieldByName(opt.Name)
+			if field.IsValid() && field.CanSet() {
+				switch field.Kind() {
+				case reflect.Float32:
+					floatVal, err := strconv.ParseFloat(val, 32)
+					if err != nil {
+						return nil, fmt.Errorf("invalid float value %s", val)
+					}
+
+					field.SetFloat(floatVal)
+				case reflect.Int:
+					intVal, err := strconv.ParseInt(val, 10, 0)
+					if err != nil {
+						return nil, fmt.Errorf("invalid int value %s", val)
+					}
+
+					field.SetInt(intVal)
+				case reflect.Bool:
+					boolVal, err := strconv.ParseBool(val)
+					if err != nil {
+						return nil, fmt.Errorf("invalid bool value %s", val)
+					}
+
+					field.SetBool(boolVal)
+				case reflect.String:
+					field.SetString(val)
+				default:
+					return nil, fmt.Errorf("unknown type %s for %s", field.Kind(), key)
+				}
+			}
+		}
+	}
+
+	bts, err := json.Marshal(opts)
+	if err != nil {
+		return nil, err
+	}
+
+	return bytes.NewReader(bts), nil
+}
+
+func getLayerDigests(layers []*LayerReader) ([]string, error) {
+	var digests []string
+	for _, l := range layers {
+		if l.Digest == "" {
+			return nil, fmt.Errorf("layer is missing a digest")
+		}
+		digests = append(digests, l.Digest)
+	}
+	return digests, nil
+}
+
+// CreateLayer creates a Layer object from a given file
+func CreateLayer(f io.ReadSeeker) (*LayerReader, error) {
+	digest, size := GetSHA256Digest(f)
+	f.Seek(0, 0)
+
+	layer := &LayerReader{
+		Layer: Layer{
+			MediaType: "application/vnd.docker.image.rootfs.diff.tar",
+			Digest:    digest,
+			Size:      size,
+		},
+		Reader: f,
+	}
+
+	return layer, nil
+}
+
+func PushModel(name, username, password string, fn func(api.ProgressResponse)) error {
+	mp := ParseModelPath(name)
+
+	fn(api.ProgressResponse{Status: "retrieving manifest"})
+
+	manifest, err := GetManifest(mp)
+	if err != nil {
+		fn(api.ProgressResponse{Status: "couldn't retrieve manifest"})
+		return err
+	}
+
+	var layers []*Layer
+	var total int
+	var completed int
+	for _, layer := range manifest.Layers {
+		layers = append(layers, layer)
+		total += layer.Size
+	}
+	layers = append(layers, &manifest.Config)
+	total += manifest.Config.Size
+
+	for _, layer := range layers {
+		exists, err := checkBlobExistence(mp, layer.Digest, username, password)
+		if err != nil {
+			return err
+		}
+
+		if exists {
+			completed += layer.Size
+			fn(api.ProgressResponse{
+				Status:    "using existing layer",
+				Digest:    layer.Digest,
+				Total:     total,
+				Completed: completed,
+			})
+			continue
+		}
+
+		fn(api.ProgressResponse{
+			Status:    "starting upload",
+			Digest:    layer.Digest,
+			Total:     total,
+			Completed: completed,
+		})
+
+		location, err := startUpload(mp, username, password)
+		if err != nil {
+			log.Printf("couldn't start upload: %v", err)
+			return err
+		}
+
+		err = uploadBlob(location, layer, username, password)
+		if err != nil {
+			log.Printf("error uploading blob: %v", err)
+			return err
+		}
+		completed += layer.Size
+		fn(api.ProgressResponse{
+			Status:    "upload complete",
+			Digest:    layer.Digest,
+			Total:     total,
+			Completed: completed,
+		})
+	}
+
+	fn(api.ProgressResponse{
+		Status:    "pushing manifest",
+		Total:     total,
+		Completed: completed,
+	})
+	url := fmt.Sprintf("%s://%s/v2/%s/manifests/%s", mp.ProtocolScheme, mp.Registry, mp.GetNamespaceRepository(), mp.Tag)
+	headers := map[string]string{
+		"Content-Type": "application/vnd.docker.distribution.manifest.v2+json",
+	}
+
+	manifestJSON, err := json.Marshal(manifest)
+	if err != nil {
+		return err
+	}
+
+	resp, err := makeRequest("PUT", url, headers, bytes.NewReader(manifestJSON), username, password)
+	if err != nil {
+		return err
+	}
+	defer resp.Body.Close()
+
+	// Check for success: For a successful upload, the Docker registry will respond with a 201 Created
+	if resp.StatusCode != http.StatusCreated {
+		body, _ := io.ReadAll(resp.Body)
+		return fmt.Errorf("registry responded with code %d: %v", resp.StatusCode, string(body))
+	}
+
+	fn(api.ProgressResponse{
+		Status:    "success",
+		Total:     total,
+		Completed: completed,
+	})
+
+	return nil
+}
+
+func PullModel(name, username, password string, fn func(api.ProgressResponse)) error {
+	mp := ParseModelPath(name)
+
+	fn(api.ProgressResponse{Status: "pulling manifest"})
+
+	manifest, err := pullModelManifest(mp, username, password)
+	if err != nil {
+		return fmt.Errorf("pull model manifest: %q", err)
+	}
+
+	var layers []*Layer
+	var total int
+	var completed int
+	for _, layer := range manifest.Layers {
+		layers = append(layers, layer)
+		total += layer.Size
+	}
+	layers = append(layers, &manifest.Config)
+	total += manifest.Config.Size
+
+	for _, layer := range layers {
+		if err := downloadBlob(mp, layer.Digest, username, password, fn); err != nil {
+			fn(api.ProgressResponse{Status: fmt.Sprintf("error downloading: %v", err), Digest: layer.Digest})
+			return err
+		}
+
+		completed += layer.Size
+	}
+
+	fn(api.ProgressResponse{Status: "writing manifest"})
+
+	manifestJSON, err := json.Marshal(manifest)
+	if err != nil {
+		return err
+	}
+
+	fp, err := mp.GetManifestPath(true)
+	if err != nil {
+		return err
+	}
+
+	err = os.WriteFile(fp, manifestJSON, 0644)
+	if err != nil {
+		log.Printf("couldn't write to %s", fp)
+		return err
+	}
+
+	fn(api.ProgressResponse{Status: "success"})
+
+	return nil
+}
+
+func pullModelManifest(mp ModelPath, username, password string) (*ManifestV2, error) {
+	url := fmt.Sprintf("%s://%s/v2/%s/manifests/%s", mp.ProtocolScheme, mp.Registry, mp.GetNamespaceRepository(), mp.Tag)
+	headers := map[string]string{
+		"Accept": "application/vnd.docker.distribution.manifest.v2+json",
+	}
+
+	resp, err := makeRequest("GET", url, headers, nil, username, password)
+	if err != nil {
+		log.Printf("couldn't get manifest: %v", err)
+		return nil, err
+	}
+	defer resp.Body.Close()
+
+	// Check for success: For a successful upload, the Docker registry will respond with a 201 Created
+	if resp.StatusCode != http.StatusOK {
+		body, _ := io.ReadAll(resp.Body)
+		return nil, fmt.Errorf("registry responded with code %d: %s", resp.StatusCode, body)
+	}
+
+	var m *ManifestV2
+	if err := json.NewDecoder(resp.Body).Decode(&m); err != nil {
+		return nil, err
+	}
+
+	return m, err
+}
+
+func createConfigLayer(layers []string) (*LayerReader, error) {
+	// TODO change architecture and OS
+	config := ConfigV2{
+		Architecture: "arm64",
+		OS:           "linux",
+		RootFS: RootFS{
+			Type:    "layers",
+			DiffIDs: layers,
+		},
+	}
+
+	configJSON, err := json.Marshal(config)
+	if err != nil {
+		return nil, err
+	}
+
+	digest, size := GetSHA256Digest(bytes.NewBuffer(configJSON))
+
+	layer := &LayerReader{
+		Layer: Layer{
+			MediaType: "application/vnd.docker.container.image.v1+json",
+			Digest:    digest,
+			Size:      size,
+		},
+		Reader: bytes.NewBuffer(configJSON),
+	}
+	return layer, nil
+}
+
+// GetSHA256Digest returns the SHA256 hash of a given buffer and returns it, and the size of buffer
+func GetSHA256Digest(r io.Reader) (string, int) {
+	h := sha256.New()
+	n, err := io.Copy(h, r)
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	return fmt.Sprintf("sha256:%x", h.Sum(nil)), int(n)
+}
+
+func startUpload(mp ModelPath, username string, password string) (string, error) {
+	url := fmt.Sprintf("%s://%s/v2/%s/blobs/uploads/", mp.ProtocolScheme, mp.Registry, mp.GetNamespaceRepository())
+
+	resp, err := makeRequest("POST", url, nil, nil, username, password)
+	if err != nil {
+		log.Printf("couldn't start upload: %v", err)
+		return "", err
+	}
+	defer resp.Body.Close()
+
+	// Check for success
+	if resp.StatusCode != http.StatusAccepted {
+		body, _ := io.ReadAll(resp.Body)
+		return "", fmt.Errorf("registry responded with code %d: %s", resp.StatusCode, body)
+	}
+
+	// Extract UUID location from header
+	location := resp.Header.Get("Location")
+	if location == "" {
+		return "", fmt.Errorf("location header is missing in response")
+	}
+
+	return location, nil
+}
+
+// Function to check if a blob already exists in the Docker registry
+func checkBlobExistence(mp ModelPath, digest string, username string, password string) (bool, error) {
+	url := fmt.Sprintf("%s://%s/v2/%s/blobs/%s", mp.ProtocolScheme, mp.Registry, mp.GetNamespaceRepository(), digest)
+
+	resp, err := makeRequest("HEAD", url, nil, nil, username, password)
+	if err != nil {
+		log.Printf("couldn't check for blob: %v", err)
+		return false, err
+	}
+	defer resp.Body.Close()
+
+	// Check for success: If the blob exists, the Docker registry will respond with a 200 OK
+	return resp.StatusCode == http.StatusOK, nil
+}
+
+func uploadBlob(location string, layer *Layer, username string, password string) error {
+	// Create URL
+	url := fmt.Sprintf("%s&digest=%s", location, layer.Digest)
+
+	headers := make(map[string]string)
+	headers["Content-Length"] = fmt.Sprintf("%d", layer.Size)
+	headers["Content-Type"] = "application/octet-stream"
+
+	// TODO change from monolithic uploads to chunked uploads
+	// TODO allow resumability
+	// TODO allow canceling uploads via DELETE
+	// TODO allow cross repo blob mount
+
+	fp, err := GetBlobsPath(layer.Digest)
+	if err != nil {
+		return err
+	}
+
+	f, err := os.Open(fp)
+	if err != nil {
+		return err
+	}
+
+	resp, err := makeRequest("PUT", url, headers, f, username, password)
+	if err != nil {
+		log.Printf("couldn't upload blob: %v", err)
+		return err
+	}
+	defer resp.Body.Close()
+
+	// Check for success: For a successful upload, the Docker registry will respond with a 201 Created
+	if resp.StatusCode != http.StatusCreated {
+		body, _ := io.ReadAll(resp.Body)
+		return fmt.Errorf("registry responded with code %d: %v", resp.StatusCode, string(body))
+	}
+
+	return nil
+}
+
+func downloadBlob(mp ModelPath, digest string, username, password string, fn func(api.ProgressResponse)) error {
+	fp, err := GetBlobsPath(digest)
+	if err != nil {
+		return err
+	}
+
+	if fi, _ := os.Stat(fp); fi != nil {
+		// we already have the file, so return
+		fn(api.ProgressResponse{
+			Digest:    digest,
+			Total:     int(fi.Size()),
+			Completed: int(fi.Size()),
+		})
+
+		return nil
+	}
+
+	var size int64
+
+	fi, err := os.Stat(fp + "-partial")
+	switch {
+	case errors.Is(err, os.ErrNotExist):
+		// noop, file doesn't exist so create it
+	case err != nil:
+		return fmt.Errorf("stat: %w", err)
+	default:
+		size = fi.Size()
+	}
+
+	url := fmt.Sprintf("%s://%s/v2/%s/blobs/%s", mp.ProtocolScheme, mp.Registry, mp.GetNamespaceRepository(), digest)
+	headers := map[string]string{
+		"Range": fmt.Sprintf("bytes=%d-", size),
+	}
+
+	resp, err := makeRequest("GET", url, headers, nil, username, password)
+	if err != nil {
+		log.Printf("couldn't download blob: %v", err)
+		return err
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusPartialContent {
+		body, _ := ioutil.ReadAll(resp.Body)
+		return fmt.Errorf("registry responded with code %d: %v", resp.StatusCode, string(body))
+	}
+
+	err = os.MkdirAll(path.Dir(fp), 0o700)
+	if err != nil {
+		return fmt.Errorf("make blobs directory: %w", err)
+	}
+
+	out, err := os.OpenFile(fp+"-partial", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644)
+	if err != nil {
+		panic(err)
+	}
+	defer out.Close()
+
+	remaining, _ := strconv.ParseInt(resp.Header.Get("Content-Length"), 10, 64)
+	completed := size
+	total := remaining + completed
+
+	for {
+		fn(api.ProgressResponse{
+			Status:    fmt.Sprintf("downloading %s", digest),
+			Digest:    digest,
+			Total:     int(total),
+			Completed: int(completed),
+		})
+
+		if completed >= total {
+			if err := os.Rename(fp+"-partial", fp); err != nil {
+				fn(api.ProgressResponse{
+					Status:    fmt.Sprintf("error renaming file: %v", err),
+					Digest:    digest,
+					Total:     int(total),
+					Completed: int(completed),
+				})
+				return err
+			}
+
+			break
+		}
+
+		n, err := io.CopyN(out, resp.Body, 8192)
+		if err != nil && !errors.Is(err, io.EOF) {
+			return err
+		}
+		completed += n
+	}
+
+	log.Printf("success getting %s\n", digest)
+	return nil
+}
+
+func makeRequest(method, url string, headers map[string]string, body io.Reader, username, password string) (*http.Response, error) {
+	req, err := http.NewRequest(method, url, body)
+	if err != nil {
+		return nil, err
+	}
+
+	for k, v := range headers {
+		req.Header.Set(k, v)
+	}
+
+	// TODO: better auth
+	if username != "" && password != "" {
+		req.SetBasicAuth(username, password)
+	}
+
+	client := &http.Client{
+		CheckRedirect: func(req *http.Request, via []*http.Request) error {
+			if len(via) >= 10 {
+				return fmt.Errorf("too many redirects")
+			}
+			log.Printf("redirected to: %s\n", req.URL)
+			return nil
+		},
+	}
+	resp, err := client.Do(req)
+	if err != nil {
+		return nil, err
+	}
+
+	return resp, nil
+}

server/modelpath.go

@@ -0,0 +1,115 @@
+package server
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+)
+
+type ModelPath struct {
+	ProtocolScheme string
+	Registry       string
+	Namespace      string
+	Repository     string
+	Tag            string
+}
+
+const (
+	DefaultRegistry       = "registry.ollama.ai"
+	DefaultNamespace      = "library"
+	DefaultTag            = "latest"
+	DefaultProtocolScheme = "https"
+)
+
+func ParseModelPath(name string) ModelPath {
+	slashParts := strings.Split(name, "/")
+	var registry, namespace, repository, tag string
+
+	switch len(slashParts) {
+	case 3:
+		registry = slashParts[0]
+		namespace = slashParts[1]
+		repository = strings.Split(slashParts[2], ":")[0]
+	case 2:
+		registry = DefaultRegistry
+		namespace = slashParts[0]
+		repository = strings.Split(slashParts[1], ":")[0]
+	case 1:
+		registry = DefaultRegistry
+		namespace = DefaultNamespace
+		repository = strings.Split(slashParts[0], ":")[0]
+	default:
+		fmt.Println("Invalid image format.")
+		return ModelPath{}
+	}
+
+	colonParts := strings.Split(name, ":")
+	if len(colonParts) == 2 {
+		tag = colonParts[1]
+	} else {
+		tag = DefaultTag
+	}
+
+	return ModelPath{
+		ProtocolScheme: DefaultProtocolScheme,
+		Registry:       registry,
+		Namespace:      namespace,
+		Repository:     repository,
+		Tag:            tag,
+	}
+}
+
+func (mp ModelPath) GetNamespaceRepository() string {
+	return fmt.Sprintf("%s/%s", mp.Namespace, mp.Repository)
+}
+
+func (mp ModelPath) GetFullTagname() string {
+	return fmt.Sprintf("%s/%s/%s:%s", mp.Registry, mp.Namespace, mp.Repository, mp.Tag)
+}
+
+func (mp ModelPath) GetShortTagname() string {
+	if mp.Registry == DefaultRegistry && mp.Namespace == DefaultNamespace {
+		return fmt.Sprintf("%s:%s", mp.Repository, mp.Tag)
+	}
+	return fmt.Sprintf("%s/%s:%s", mp.Namespace, mp.Repository, mp.Tag)
+}
+
+func (mp ModelPath) GetManifestPath(createDir bool) (string, error) {
+	home, err := os.UserHomeDir()
+	if err != nil {
+		return "", err
+	}
+
+	path := filepath.Join(home, ".ollama", "models", "manifests", mp.Registry, mp.Namespace, mp.Repository, mp.Tag)
+	if createDir {
+		if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+			return "", err
+		}
+	}
+
+	return path, nil
+}
+
+func GetManifestPath() (string, error) {
+	home, err := os.UserHomeDir()
+	if err != nil {
+		return "", err
+	}
+
+	return filepath.Join(home, ".ollama", "models", "manifests"), nil
+}
+
+func GetBlobsPath(digest string) (string, error) {
+	home, err := os.UserHomeDir()
+	if err != nil {
+		return "", err
+	}
+
+	path := filepath.Join(home, ".ollama", "models", "blobs", digest)
+	if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+		return "", err
+	}
+
+	return path, nil
+}

server/models.go

@@ -1,128 +0,0 @@
-package server
-
-import (
-	"encoding/json"
-	"errors"
-	"fmt"
-	"io"
-	"net/http"
-	"os"
-	"path"
-	"strconv"
-)
-
-const directoryURL = "https://ollama.ai/api/models"
-
-type Model struct {
-	Name             string `json:"name"`
-	DisplayName      string `json:"display_name"`
-	Parameters       string `json:"parameters"`
-	URL              string `json:"url"`
-	ShortDescription string `json:"short_description"`
-	Description      string `json:"description"`
-	PublishedBy      string `json:"published_by"`
-	OriginalAuthor   string `json:"original_author"`
-	OriginalURL      string `json:"original_url"`
-	License          string `json:"license"`
-}
-
-func (m *Model) FullName() string {
-	home, err := os.UserHomeDir()
-	if err != nil {
-		panic(err)
-	}
-
-	return path.Join(home, ".ollama", "models", m.Name+".bin")
-}
-
-func (m *Model) TempFile() string {
-	fullName := m.FullName()
-	return path.Join(
-		path.Dir(fullName),
-		fmt.Sprintf(".%s.part", path.Base(fullName)),
-	)
-}
-
-func getRemote(model string) (*Model, error) {
-	// resolve the model download from our directory
-	resp, err := http.Get(directoryURL)
-	if err != nil {
-		return nil, fmt.Errorf("failed to get directory: %w", err)
-	}
-	defer resp.Body.Close()
-	body, err := io.ReadAll(resp.Body)
-	if err != nil {
-		return nil, fmt.Errorf("failed to read directory: %w", err)
-	}
-	var models []Model
-	err = json.Unmarshal(body, &models)
-	if err != nil {
-		return nil, fmt.Errorf("failed to parse directory: %w", err)
-	}
-	for _, m := range models {
-		if m.Name == model {
-			return &m, nil
-		}
-	}
-	return nil, fmt.Errorf("model not found in directory: %s", model)
-}
-
-func saveModel(model *Model, fn func(total, completed int64)) error {
-	// this models cache directory is created by the server on startup
-
-	client := &http.Client{}
-	req, err := http.NewRequest("GET", model.URL, nil)
-	if err != nil {
-		return fmt.Errorf("failed to download model: %w", err)
-	}
-
-	var size int64
-
-	// completed file doesn't exist, check partial file
-	fi, err := os.Stat(model.TempFile())
-	switch {
-	case errors.Is(err, os.ErrNotExist):
-		// noop, file doesn't exist so create it
-	case err != nil:
-		return fmt.Errorf("stat: %w", err)
-	default:
-		size = fi.Size()
-	}
-
-	req.Header.Add("Range", fmt.Sprintf("bytes=%d-", size))
-
-	resp, err := client.Do(req)
-	if err != nil {
-		return fmt.Errorf("failed to download model: %w", err)
-	}
-	defer resp.Body.Close()
-
-	if resp.StatusCode >= 400 {
-		return fmt.Errorf("failed to download model: %s", resp.Status)
-	}
-
-	out, err := os.OpenFile(model.TempFile(), os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644)
-	if err != nil {
-		panic(err)
-	}
-	defer out.Close()
-
-	remaining, _ := strconv.ParseInt(resp.Header.Get("Content-Length"), 10, 64)
-	completed := size
-
-	total := remaining + completed
-
-	for {
-		fn(total, completed)
-		if completed >= total {
-			return os.Rename(model.TempFile(), model.FullName())
-		}
-
-		n , err := io.CopyN(out, resp.Body, 8192)
-		if err != nil && !errors.Is(err, io.EOF) {
-			return err
-		}
-
-		completed += n
-	}
-}

server/routes.go

@@ -1,104 +1,94 @@
 package server
 
 import (
-	"embed"
 	"encoding/json"
-	"errors"
 	"io"
 	"log"
-	"math"
 	"net"
 	"net/http"
 	"os"
-	"path"
+	"path/filepath"
 	"strings"
 	"text/template"
 	"time"
 
+	"dario.cat/mergo"
 	"github.com/gin-gonic/gin"
-	"github.com/lithammer/fuzzysearch/fuzzy"
 
 	"github.com/jmorganca/ollama/api"
 	"github.com/jmorganca/ollama/llama"
 )
 
-//go:embed templates/*
-var templatesFS embed.FS
-var templates = template.Must(template.ParseFS(templatesFS, "templates/*.prompt"))
-
 func cacheDir() string {
 	home, err := os.UserHomeDir()
 	if err != nil {
 		panic(err)
 	}
 
-	return path.Join(home, ".ollama")
+	return filepath.Join(home, ".ollama")
 }
 
 func generate(c *gin.Context) {
 	start := time.Now()
 
-	req := api.GenerateRequest{
-		Options: api.DefaultOptions(),
-	}
-
+	var req api.GenerateRequest
 	if err := c.ShouldBindJSON(&req); err != nil {
 		c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
 		return
 	}
 
-	if remoteModel, _ := getRemote(req.Model); remoteModel != nil {
-		req.Model = remoteModel.FullName()
-	}
-	if _, err := os.Stat(req.Model); err != nil {
-		if !errors.Is(err, os.ErrNotExist) {
-			c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
-			return
-		}
-		req.Model = path.Join(cacheDir(), "models", req.Model+".bin")
+	model, err := GetModel(req.Model)
+	if err != nil {
+		c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
+		return
 	}
 
-	ch := make(chan any)
-	go stream(c, ch)
-
-	templateNames := make([]string, 0, len(templates.Templates()))
-	for _, template := range templates.Templates() {
-		templateNames = append(templateNames, template.Name())
+	opts := api.DefaultOptions()
+	if err := mergo.Merge(&opts, model.Options, mergo.WithOverride); err != nil {
+		c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+		return
 	}
 
-	match, _ := matchRankOne(path.Base(req.Model), templateNames)
-	if template := templates.Lookup(match); template != nil {
-		var sb strings.Builder
-		if err := template.Execute(&sb, req); err != nil {
-			c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
-			return
-		}
-
-		req.Prompt = sb.String()
+	if err := mergo.Merge(&opts, req.Options, mergo.WithOverride); err != nil {
+		c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+		return
 	}
 
-	llm, err := llama.New(req.Model, req.Options)
+	templ, err := template.New("").Parse(model.Prompt)
+	if err != nil {
+		c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+		return
+	}
+
+	var sb strings.Builder
+	if err = templ.Execute(&sb, req); err != nil {
+		c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+		return
+	}
+	req.Prompt = sb.String()
+
+	llm, err := llama.New(model.ModelPath, opts)
 	if err != nil {
 		c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
 		return
 	}
 	defer llm.Close()
 
-	fn := func(r api.GenerateResponse) {
-		r.Model = req.Model
-		r.CreatedAt = time.Now().UTC()
-		if r.Done {
-			r.TotalDuration = time.Since(start)
-		}
+	ch := make(chan any)
+	go func() {
+		defer close(ch)
+		llm.Predict(req.Context, req.Prompt, func(r api.GenerateResponse) {
+			r.Model = req.Model
+			r.CreatedAt = time.Now().UTC()
+			if r.Done {
+				r.TotalDuration = time.Since(start)
+			}
 
-		ch <- r
-	}
-
-	if err := llm.Predict(req.Prompt, fn); err != nil {
-		c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
-		return
-	}
+			ch <- r
+		})
+	}()
 
+	streamResponse(c, ch)
 }
 
 func pull(c *gin.Context) {
@@ -108,45 +98,123 @@ func pull(c *gin.Context) {
 		return
 	}
 
-	remote, err := getRemote(req.Model)
-	if err != nil {
-		c.JSON(http.StatusBadGateway, gin.H{"error": err.Error()})
-		return
-	}
+	ch := make(chan any)
+	go func() {
+		defer close(ch)
+		fn := func(r api.ProgressResponse) {
+			ch <- r
+		}
 
-	// check if completed file exists
-	fi, err := os.Stat(remote.FullName())
-	switch {
-	case errors.Is(err, os.ErrNotExist):
-		// noop, file doesn't exist so create it
-	case err != nil:
-		c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
-		return
-	default:
-		c.JSON(http.StatusOK, api.PullProgress{
-			Total:     fi.Size(),
-			Completed: fi.Size(),
-			Percent:   100,
-		})
+		if err := PullModel(req.Name, req.Username, req.Password, fn); err != nil {
+			c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+			return
+		}
+	}()
 
+	streamResponse(c, ch)
+}
+
+func push(c *gin.Context) {
+	var req api.PushRequest
+	if err := c.ShouldBindJSON(&req); err != nil {
+		c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
 		return
 	}
 
 	ch := make(chan any)
-	go stream(c, ch)
-
-	fn := func(total, completed int64) {
-		ch <- api.PullProgress{
-			Total:     total,
-			Completed: completed,
-			Percent:   float64(completed) / float64(total) * 100,
+	go func() {
+		defer close(ch)
+		fn := func(r api.ProgressResponse) {
+			ch <- r
 		}
+
+		if err := PushModel(req.Name, req.Username, req.Password, fn); err != nil {
+			c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+			return
+		}
+	}()
+
+	streamResponse(c, ch)
+}
+
+func create(c *gin.Context) {
+	var req api.CreateRequest
+	if err := c.ShouldBindJSON(&req); err != nil {
+		c.JSON(http.StatusBadRequest, gin.H{"message": err.Error()})
+		return
 	}
 
-	if err := saveModel(remote, fn); err != nil {
+	// NOTE consider passing the entire Modelfile in the json instead of the path to it
+
+	file, err := os.Open(req.Path)
+	if err != nil {
+		c.JSON(http.StatusBadRequest, gin.H{"message": err.Error()})
+		return
+	}
+	defer file.Close()
+
+	ch := make(chan any)
+	go func() {
+		defer close(ch)
+		fn := func(status string) {
+			ch <- api.CreateProgress{
+				Status: status,
+			}
+		}
+
+		if err := CreateModel(req.Name, file, fn); err != nil {
+			c.JSON(http.StatusBadRequest, gin.H{"message": err.Error()})
+			return
+		}
+	}()
+
+	streamResponse(c, ch)
+}
+
+func list(c *gin.Context) {
+	var models []api.ListResponseModel
+	fp, err := GetManifestPath()
+	if err != nil {
 		c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
 		return
 	}
+	err = filepath.Walk(fp, func(path string, info os.FileInfo, err error) error {
+		if err != nil {
+			return err
+		}
+		if !info.IsDir() {
+			fi, err := os.Stat(path)
+			if err != nil {
+				log.Printf("skipping file: %s", fp)
+				return nil
+			}
+			path := path[len(fp)+1:]
+			slashIndex := strings.LastIndex(path, "/")
+			if slashIndex == -1 {
+				return nil
+			}
+			tag := path[:slashIndex] + ":" + path[slashIndex+1:]
+			mp := ParseModelPath(tag)
+			manifest, err := GetManifest(mp)
+			if err != nil {
+				log.Printf("skipping file: %s", fp)
+				return nil
+			}
+			model := api.ListResponseModel{
+				Name:       mp.GetShortTagname(),
+				Size:       manifest.GetTotalSize(),
+				ModifiedAt: fi.ModTime(),
+			}
+			models = append(models, model)
+		}
+		return nil
+	})
+	if err != nil {
+		c.JSON(http.StatusInternalServerError, gin.H{"error": err.Error()})
+		return
+	}
+
+	c.JSON(http.StatusOK, api.ListResponse{models})
 }
 
 func Serve(ln net.Listener) error {
@@ -158,6 +226,9 @@ func Serve(ln net.Listener) error {
 
 	r.POST("/api/pull", pull)
 	r.POST("/api/generate", generate)
+	r.POST("/api/create", create)
+	r.POST("/api/push", push)
+	r.GET("/api/tags", list)
 
 	log.Printf("Listening on %s", ln.Addr())
 	s := &http.Server{
@@ -167,19 +238,7 @@ func Serve(ln net.Listener) error {
 	return s.Serve(ln)
 }
 
-func matchRankOne(source string, targets []string) (bestMatch string, bestRank int) {
-	bestRank = math.MaxInt
-	for _, target := range targets {
-		if rank := fuzzy.LevenshteinDistance(source, target); bestRank > rank {
-			bestRank = rank
-			bestMatch = target
-		}
-	}
-
-	return
-}
-
-func stream(c *gin.Context, ch chan any) {
+func streamResponse(c *gin.Context, ch chan any) {
 	c.Stream(func(w io.Writer) bool {
 		val, ok := <-ch
 		if !ok {

server/templates/alpaca.prompt

@@ -1,4 +1,6 @@
+{{- if not .Context }}
 Below is an instruction that describes a task. Write a response that appropriately completes the request.
+{{- end }}
 
 ### Instruction:
 {{ .Prompt }}

server/templates/falcon.prompt

@@ -1,3 +1,5 @@
+{{- if not .Context }}
 A helpful assistant who helps the user with any questions asked.
+{{- end }}
 User: {{ .Prompt }}
 Assistant:

server/templates/mpt.prompt

@@ -1,4 +1,6 @@
+{{- if not .Context }}
 Below is an instruction that describes a task. Write a response that appropriately completes the request. Be concise. Once the request is completed, include no other text.
+{{- end }}
 ### Instruction:
 {{ .Prompt }}
 ### Response:

server/templates/orca.prompt

@@ -1,5 +1,7 @@
+{{- if not .Context }}
 ### System:
 You are an AI assistant that follows instruction extremely well. Help as much as you can.
+{{- end }}
 
 ### User:
 {{ .Prompt }}

server/templates/vicuna.prompt

@@ -1,4 +1,6 @@
+{{ if not .Context }}
 A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.
+{{- end }}
 
 USER: {{ .Prompt }}
 ASSISTANT:

server/templates/wizardcoder.prompt

@@ -1,4 +1,6 @@
+{{- if not .Context }}
 Below is an instruction that describes a task. Write a response that appropriately completes the request
+{{- end }}
 
 ### Instruction: {{ .Prompt }}
 

web/app/download/page.tsx

@@ -1,3 +1,6 @@
+import Image from 'next/image'
+
+import Header from '../header'
 import Downloader from './downloader'
 import Signup from './signup'
 
@@ -26,22 +29,19 @@ export default async function Download() {
   }
 
   return (
-    <main className='flex min-h-screen max-w-2xl flex-col p-4 lg:p-24 items-center mx-auto'>
-      <img src='/ollama.png' className='w-16 h-auto' />
-      <section className='my-12 text-center'>
-        <h2 className='my-2 max-w-md text-3xl tracking-tight'>Downloading Ollama</h2>
-        <h3 className='text-sm text-neutral-500'>
-          Problems downloading?{' '}
-          <a href={asset.browser_download_url} className='underline'>
-            Try again
-          </a>
-        </h3>
-        <Downloader url={asset.browser_download_url} />
-      </section>
-      <section className='max-w-sm flex flex-col w-full items-center border border-neutral-200 rounded-xl px-8 pt-8 pb-2'>
-        <p className='text-lg leading-tight text-center mb-6 max-w-[260px]'>Sign up for updates</p>
+    <>
+      <Header />
+      <main className='flex min-h-screen max-w-6xl flex-col py-20 px-16 lg:p-32 items-center mx-auto'>
+        <Image src='/ollama.png' width={64} height={64} alt='ollamaIcon' />
+        <section className='mt-12 mb-8 text-center'>
+          <h2 className='my-2 max-w-md text-3xl tracking-tight'>Downloading...</h2>
+          <h3 className='text-base text-neutral-500 mt-12 max-w-[16rem]'>
+            While Ollama downloads, sign up to get notified of new updates.
+          </h3>
+          <Downloader url={asset.browser_download_url} />
+        </section>
         <Signup />
-      </section>
-    </main>
+      </main>
+    </>
   )
 }

web/app/download/signup.tsx

@@ -28,7 +28,7 @@ export default function Signup() {
 
         return false
       }}
-      className='flex self-stretch flex-col gap-3 h-32'
+      className='flex self-stretch flex-col gap-3 h-32 md:mx-40 lg:mx-72'
     >
       <input
         required
@@ -37,13 +37,13 @@ export default function Signup() {
         onChange={e => setEmail(e.target.value)}
         type='email'
         placeholder='your@email.com'
-        className='bg-neutral-100 rounded-lg px-4 py-2 focus:outline-none placeholder-neutral-500'
+        className='border border-neutral-200 rounded-lg px-4 py-2 focus:outline-none placeholder-neutral-300'
       />
       <input
         type='submit'
         value='Get updates'
         disabled={submitting}
-        className='bg-black text-white disabled:text-neutral-200 disabled:bg-neutral-700 rounded-lg px-4 py-2 focus:outline-none cursor-pointer'
+        className='bg-black text-white disabled:text-neutral-200 disabled:bg-neutral-700 rounded-full px-4 py-2 focus:outline-none cursor-pointer'
       />
       {success && <p className='text-center text-sm'>You&apos;re signed up for updates</p>}
     </form>

web/app/header.tsx

@@ -0,0 +1,26 @@
+import Link from "next/link"
+
+const navigation = [
+  { name: 'Discord', href: 'https://discord.gg/MrfB5FbNWN' },
+  { name: 'Github', href: 'https://github.com/jmorganca/ollama' },
+  { name: 'Download', href: '/download' },
+]
+
+export default function Header() {  
+  return (
+    <header className="absolute inset-x-0 top-0 z-50">
+      <nav className="mx-auto flex items-center justify-between px-10 py-4">        
+        <Link className="flex-1 font-bold" href="/">
+          Ollama
+        </Link>
+        <div className="flex space-x-8">
+          {navigation.map((item) => (
+            <Link key={item.name} href={item.href} className="text-sm leading-6 text-gray-900">
+              {item.name}
+            </Link>
+          ))}
+        </div>
+      </nav>
+    </header >
+  )
+}

web/app/page.tsx

@@ -1,34 +1,32 @@
-import { AiFillApple } from 'react-icons/ai'
+import Image from 'next/image'
+import Link from 'next/link'
 
-import models from '../../models.json'
+import Header from './header'
 
 export default async function Home() {
   return (
-    <main className='flex min-h-screen max-w-2xl flex-col p-4 lg:p-24'>
-      <img src='/ollama.png' className='w-16 h-auto' />
-      <section className='my-4'>
-        <p className='my-3 max-w-md'>
-          <a className='underline' href='https://github.com/jmorganca/ollama'>
-            Ollama
-          </a>{' '}
-          is a tool for running large language models, currently for macOS with Windows and Linux coming soon.
-          <br />
-          <br />
-          <a href='/download'>
-            <button className='bg-black text-white text-sm py-2 px-3 rounded-lg flex items-center gap-2'>
-              <AiFillApple className='h-auto w-5 relative -top-px' /> Download for macOS
-            </button>
-          </a>
-        </p>
-      </section>
-      <section className='my-4'>
-        <h2 className='mb-4 text-lg'>Example models you can try running:</h2>
-        {models.map(m => (
-          <div className='my-2 grid font-mono' key={m.name}>
-            <code className='py-0.5'>ollama run {m.name}</code>
+    <>
+      <Header />
+      <main className='flex min-h-screen max-w-6xl flex-col py-20 px-16 md:p-32 items-center mx-auto'>
+        <Image src='/ollama.png' width={64} height={64} alt='ollamaIcon' />
+        <section className='my-12 text-center'>
+          <div className='flex flex-col space-y-2'>
+            <h2 className='md:max-w-[18rem] mx-auto my-2 text-3xl tracking-tight'>Portable large language models</h2>
+            <h3 className='md:max-w-xs mx-auto text-base text-neutral-500'>
+              Bundle a model’s weights, configuration, prompts, data and more into self-contained packages that run anywhere.
+            </h3>
           </div>
-        ))}
-      </section>
-    </main>
+          <div className='mx-auto flex flex-col space-y-4 mt-12'>
+            <Link href='/download' className='md:mx-10 lg:mx-14 bg-black text-white rounded-full px-4 py-2 focus:outline-none cursor-pointer'>
+              Download
+            </Link>
+            <p className='text-neutral-500 text-sm '>
+            Available for macOS with Apple Silicon <br />
+            Windows & Linux support coming soon.
+            </p>
+          </div>
+        </section>
+      </main>
+    </>
   )
 }