New Container: OllamaContainer #617

Activity

1. bricefotzo

   added

   🚀 enhancement

   on Jun 26, 2024
2. alexanderankin commented on Jun 26, 2024

   

   alexanderankin

   on Jun 26, 2024

   Member

   More actions

   does it make sense to mount a volume instead of/as well as committing a container?
3. bricefotzo commented on Jun 27, 2024

   

   bricefotzo

   on Jun 27, 2024

   Last edited by bricefotzo

   ContributorAuthor

   More actions

   @alexanderankin you raise a good point!

   I asked myself the same question but as it's implemented with commit in the java and typescript versions I thought it's maybe because it's simpler and more robust to implement with commit.

   Using commit, you don't need to identify all the files generated by ollama to make the volume binding where using volume you should specify all the needed files. The Ollama PATH is /root/.ollama but is there another locations affected by the pull of a model? I don't know, so having the doubt, the commit approach looks safer.

   Plus, the commit approach ensures that the test environment is completely self-contained within the Docker image, which can be beneficial for reproducibility and portability across different test environments I guess?

   I think @ilopezluna or @eddumelendez have a better answer to your question
4. alexanderankin commented on Jun 27, 2024

   

   alexanderankin

   on Jun 27, 2024

   Member

   More actions

   I have also realized that this is a more general concern which would also apply to java implementation. Looking at the various available images for ollama as well. one theory is that sizes vary based on number of gpu drivers available, so macos build is quite small - but then using this image on a mac would be different, so thinking about how to test this as well... this may be one of those images that only works on a linux machine, which may be okay.
5. alexanderankin

   closed this as completedin 5442d05on Jun 27, 2024
6. alexanderankin

   closed this as completedin #618on Jun 27, 2024
7. alexanderankin commented on Jun 27, 2024

   

   alexanderankin

   on Jun 27, 2024

   Member

   More actions

   I added an ollama_dir option, which maybe should have been named ollama_home or something but it does seem to work (there is a test as well)
8. bricefotzo commented on Jun 27, 2024

   

   bricefotzo

   on Jun 27, 2024

   ContributorAuthor

   More actions

   Nice idea, so that both are possible! Thanks @alexanderankin ! Can't wait to try
9. alexanderankin commented on Jun 27, 2024

   

   alexanderankin

   on Jun 27, 2024

   Member

   More actions

   why wait:

   mkdir testcontainers-python-617 && cd $_
   python -m venv .venv && source $_/bin/activate
   pip install git+https://github.com/testcontainers/testcontainers-python@main

   script.py:

   from json import loads
   from pathlib import Path
   
   from requests import post
   from testcontainers.ollama import OllamaContainer
   
   
   def split_by_line(generator):
       data = b''
       for each_item in generator:
           for line in each_item.splitlines(True):
               data += line
               if data.endswith((b'\r\r', b'\n\n', b'\r\n\r\n', b'\n')):
                   yield from data.splitlines()
                   data = b''
       if data:
           yield from data.splitlines()
   
   
   with OllamaContainer(ollama_home=Path.home() / ".ollama") as ollama:
       if "llama3:latest" not in [e["name"] for e in ollama.list_models()]:
           print("did not find 'llama3:latest', pulling")
           ollama.pull_model("llama3:latest")
       endpoint = ollama.get_endpoint()
       for chunk in split_by_line(
               post(url=f"{endpoint}/api/chat", stream=True, json={
                   "model": "llama3:latest",
                   "messages": [{"role": "user", "content": "what color is the sky?"}]
               })
       ):
           print(loads(chunk)["message"]["content"], end="")

   🎉React with 🎉1Reacted by Brice Fotzo
10. alexanderankin

    added a commit that references this issue on Jun 28, 2024

    feat(core): Add support for ollama module (testcontainers#618)

    ...

    This issue will close once commit 5442d05 is merged into the 'main' branch.

    5442d05