mirror of
https://github.com/mealie-recipes/mealie.git
synced 2026-05-25 19:20:26 -04:00
feat: In-app AI Provider Configuration (#7650)
This commit is contained in:
Michael Genson
GitHub
@@ -1,10 +1,10 @@
|
||||
!!! info
|
||||
This guide was submitted by a community member. Find something wrong? Submit a PR to get it fixed!
|
||||
|
||||
An easy way to add recipes to Mealie from an Apple device is via an Apple Shortcut. This is a short guide to install an configure a shortcut able to add recipes via a link or image(s).
|
||||
An easy way to add recipes to Mealie from an Apple device is via an Apple Shortcut. This is a short guide to install an configure a shortcut able to add recipes via a link or image(s).
|
||||
|
||||
!!! note
|
||||
If adding via images make sure to enable [Mealie's OpenAI Integration](https://docs.mealie.io/documentation/getting-started/installation/open-ai/)
|
||||
If adding via images make sure to enable [Mealie's AI Integration](https://docs.mealie.io/documentation/getting-started/installation/ai-providers)
|
||||
|
||||
## Javascript can only be run via Shortcuts on the Safari browser on MacOS and iOS. If you do not use Safari you may skip this section
|
||||
Some sites have begun blocking AI scraping bots, inadvertently blocking the recipe scraping library Mealie uses as well. To circumvent this, the shortcut uses javascript to capture the raw html loaded in the browser and sends that to mealie when possible.
|
||||
@@ -23,7 +23,7 @@ An API key is needed to authenticate with mealie. To create an api key for a use
|
||||
The shortcut can be installed via **[This link](https://www.icloud.com/shortcuts/52834724050b42aebe0f2efd8d067360)**. Upon install, replace "MEALIE_API_KEY" with the API key generated previously and "MEALIE_URI" with the full URL used to access your mealie instance e.g. "http://10.0.0.5:9000" or "https://mealie.domain.com".
|
||||
|
||||
## Using the Shortcut
|
||||
Once installed, the shortcut will automatically appear as an option when sharing an image or webpage. It can also be useful to add the shortcut to the home screen of your device. If selected from the home screen or shortcuts app, a menu will appear with prompts to import via **taking photo(s)**, **selecting photo(s)**, **scanning a URL**, or **pasting a URL**.
|
||||
Once installed, the shortcut will automatically appear as an option when sharing an image or webpage. It can also be useful to add the shortcut to the home screen of your device. If selected from the home screen or shortcuts app, a menu will appear with prompts to import via **taking photo(s)**, **selecting photo(s)**, **scanning a URL**, or **pasting a URL**.
|
||||
|
||||
!!! note
|
||||
Despite the Mealie API being able to accept multiple recipe images for import it is currently impossible to send multiple files in 1 web request via Shortcuts. Instead, the shortcut combines the images into a singular, vertically-concatenated image to send to mealie. This can result in slightly less-accurate text recognition.
|
||||
Despite the Mealie API being able to accept multiple recipe images for import it is currently impossible to send multiple files in 1 web request via Shortcuts. Instead, the shortcut combines the images into a singular, vertically-concatenated image to send to mealie. This can result in slightly less-accurate text recognition.
|
||||
|
||||
@@ -33,7 +33,7 @@
|
||||
6. Click the Edit button/icon again
|
||||
7. Scroll to the ingredients and you should see new fields for Amount, Unit, Food, and Note. The Note in particular will contain the original text of the Recipe.
|
||||
8. Click `Parse` and you will be taken to the ingredient parsing page.
|
||||
9. Choose your parser. The `Natural Language Parser` works very well, but you can also use the `Brute Parser`, or the `OpenAI Parser` if you've [enabled OpenAI support](./installation/backend-config.md#openai).
|
||||
9. Choose your parser. The `Natural Language Parser` works very well, but you can also use the `Brute Parser`, or the `OpenAI Parser` if you've [enabled AI support](./installation/ai-providers.md).
|
||||
10. Click `Parse All`, and your ingredients should be separated out into Units and Foods based on your seeding in Step 1 above.
|
||||
11. For ingredients where the Unit or Food was not found, you can click a button to accept an automatically suggested Food to add to the database. Or, manually enter the Unit/Food and hit `Enter` (or click `Create`) to add it to the database
|
||||
12. When done, click `Save All` and you will be taken back to the recipe. Now the Unit and Food fields of the recipe should be filled out.
|
||||
|
||||
@@ -11,7 +11,7 @@ Mealie offers several ways to create recipes:
|
||||
- **Recipe HTML or JSON:** Copy/paste structured HTML or JSON and Mealie can import it.
|
||||
- **Manual Editor:** Create recipes from scratch using the integrated editor.
|
||||
|
||||
Mealie's [AI integration](./installation/open-ai.md) greatly expands the ways you can create recipes:
|
||||
Mealie's [AI integration](./installation/ai-providers.md) greatly expands the ways you can create recipes:
|
||||
|
||||
- **Image Import:** Upload an image of a written or typed recipe and Mealie will use OCR and AI to import it.
|
||||
- **Video URL Import:** Provide a video URL (e.g., YouTube) and Mealie will transcribe the audio and turn it into a recipe.
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
# AI Integration
|
||||
|
||||
:octicons-tag-24: v1.7.0
|
||||
|
||||
Mealie's AI integration enables several features and enhancements throughout the application. To enable AI features, you must have access to an AI provider (such as OpenAI). Mealie works with any OpenAI-compatible API.
|
||||
|
||||
## Configuration
|
||||
|
||||
To set up AI providers, visit your group settings.
|
||||
|
||||
[Group Settings Demo](https://demo.mealie.io/group){ .md-button .md-button--primary }
|
||||
|
||||
- To enable AI features at all, you *must* set a default provider (e.g. `gpt-5`)
|
||||
- To enable image recognition features, such as creating a recipe from an image, configure a provider capable of image recognition (e.g. `gpt-5`)
|
||||
- To enable audio transcription features, such as importing a recipe from a video, configure a provider capable of audio transcriptions (e.g. `whisper-1`)
|
||||
|
||||
For most users, choosing an OpenAI model (such as `gpt-5`) and supplying the OpenAI API key is all you need to do. Note that while OpenAI has a free tier, it's not sufficiently capable for Mealie (or most other production use cases). For more information, check out [OpenAI's rate limits](https://platform.openai.com/docs/guides/rate-limits). If you deposit $5 into your OpenAI account, you will be permanently bumped up to Tier 1, which is sufficient for Mealie. Cost per-request is dependant on many factors, but Mealie tries to keep token counts conservative.
|
||||
|
||||
If you have another provider you'd like to use, such as Azure, you can configure Mealie to use that instead as long as it has an OpenAI-compatible API. For instance, a common self-hosted alternative to OpenAI is [Ollama](https://ollama.com/). To use Ollama with Mealie, set your `base_url` to `http://localhost:11434/v1` (where `http://localhost:11434` is wherever you're hosting Ollama, and `/v1` enables the OpenAI-compatible endpoints). Note that you *must* provide an API key, even though it is ultimately ignored by Ollama.
|
||||
|
||||
Note that some models are capable of handling multiple features (e.g. `gpt-5` can handle both normal chat requests and image recognition requests). You may configure one provider for multiple provider features.
|
||||
|
||||
While Mealie has prompts for each AI task, you can override these with your own prompts if you'd like. For more information, check out the [backend configuration](./installation/ai-providers.md).
|
||||
|
||||
## AI Features
|
||||
- The OpenAI Ingredient Parser can be used as an alternative to the NLP and Brute Force parsers. Simply choose the OpenAI parser while parsing ingredients (:octicons-tag-24: v1.7.0)
|
||||
- When importing a recipe via URL, if the default recipe scraper is unable to read the recipe data from a webpage, the webpage contents will be parsed by OpenAI (:octicons-tag-24: v1.9.0)
|
||||
- You can import an image of a written recipe, which is sent to OpenAI and imported into Mealie. The recipe can be hand-written or typed, as long as the text is in the picture. You can also optionally have OpenAI translate the recipe into your own language (:octicons-tag-24: v1.12.0)
|
||||
- You can import a recipe via a video URL (e.g., a YouTube link). The video is transcribed AI, and the transcription is parsed into a recipe (:octicons-tag-24: v3.13.0)
|
||||
@@ -120,22 +120,10 @@ For usage, see [Usage - OpenID Connect](../authentication/oidc-v2.md)
|
||||
|
||||
:octicons-tag-24: v1.7.0
|
||||
|
||||
Mealie supports various integrations using OpenAI. For more information, check out our [OpenAI documentation](./open-ai.md).
|
||||
For custom mapping variables (e.g. OPENAI_CUSTOM_HEADERS) you should pass values as JSON encoded strings (e.g. `OPENAI_CUSTOM_PARAMS='{"k1": "v1", "k2": "v2"}'`)
|
||||
Mealie supports various integrations using OpenAI. For more information, check out our [OpenAI documentation](./ai-providers.md).
|
||||
|
||||
| Variables | Default | Description |
|
||||
|-------------------------------------------------------------------------|:-----------:|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| OPENAI_BASE_URL<super>[†][secrets]</super> | None | The base URL for the OpenAI API. If you're not sure, leave this empty to use the standard OpenAI platform |
|
||||
| OPENAI_API_KEY<super>[†][secrets]</super> | None | Your OpenAI API Key. Enables OpenAI-related features |
|
||||
| OPENAI_MODEL | gpt-4o | Which OpenAI model to use. If you're not sure, leave this empty |
|
||||
| OPENAI_AUDIO_MODEL <br/> :octicons-tag-24: v3.13.0 | whisper-1 | Which OpenAI model to use for audio transcriptions, if enabled. If you're not sure, leave this empty |
|
||||
| OPENAI_CUSTOM_HEADERS <br/> :octicons-tag-24: v2.0.0 | None | Custom HTTP headers to add to all OpenAI requests. This should generally be left empty unless your custom service requires them |
|
||||
| OPENAI_CUSTOM_PARAMS <br/> :octicons-tag-24: v2.0.0 | None | Custom HTTP query params to add to all OpenAI requests. This should generally be left empty unless your custom service requires them |
|
||||
| OPENAI_ENABLE_IMAGE_SERVICES <br/> :octicons-tag-24: v1.12.0 | True | Whether to enable OpenAI image services, such as creating recipes via image. Leave this enabled unless your custom model doesn't support it, or you want to reduce costs |
|
||||
| OPENAI_ENABLE_TRANSCRIPTION_SERVICES <br/> :octicons-tag-24: v3.13.0 | True | Whether to enable OpenAI transcription services, such as creating recipes via video URL. Leave this enabled unless your custom model doesn't support it, or you want to reduce costs |
|
||||
| OPENAI_WORKERS | 2 | Number of OpenAI workers per request. Higher values may increase processing speed, but will incur additional API costs |
|
||||
| OPENAI_SEND_DATABASE_DATA | True | Whether to send Mealie data to OpenAI to improve request accuracy. This will incur additional API costs |
|
||||
| OPENAI_REQUEST_TIMEOUT | 300 | The number of seconds to wait for an OpenAI request to complete before cancelling the request. Leave this empty unless you're running into timeout issues on slower hardware |
|
||||
| OPENAI_CUSTOM_PROMPT_DIR <br/> :octicons-tag-24: v3.10.0 | None. | Path to custom prompt files. Only existing files in your custom directory will override the defaults; any missing or empty custom files will automatically fall back to the system defaults. See https://github.com/mealie-recipes/mealie/tree/mealie-next/mealie/services/openai/prompts for expected file names. |
|
||||
|
||||
### Theming
|
||||
@@ -315,7 +303,6 @@ at least these sensitive environment variables when working within shared enviro
|
||||
- `POSTGRES_PASSWORD`
|
||||
- `SMTP_PASSWORD`
|
||||
- `LDAP_QUERY_PASSWORD`
|
||||
- `OPENAI_API_KEY`
|
||||
|
||||
[docker-secrets]: https://docs.docker.com/compose/use-secrets/
|
||||
[secrets]: #docker-secrets
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
# OpenAI Integration
|
||||
|
||||
:octicons-tag-24: v1.7.0
|
||||
|
||||
Mealie's OpenAI integration enables several features and enhancements throughout the application. To enable OpenAI features, you must have an account with OpenAI and configure Mealie to use the OpenAI API key (for more information, check out the [backend configuration](./backend-config.md#openai)).
|
||||
|
||||
## Configuration
|
||||
|
||||
For most users, supplying the OpenAI API key is all you need to do; you will use the regular OpenAI service with the default language model. Note that while OpenAI has a free tier, it's not sufficiently capable for Mealie (or most other production use cases). For more information, check out [OpenAI's rate limits](https://platform.openai.com/docs/guides/rate-limits). If you deposit $5 into your OpenAI account, you will be permanently bumped up to Tier 1, which is sufficient for Mealie. Cost per-request is dependant on many factors, but Mealie tries to keep token counts conservative.
|
||||
|
||||
Alternatively, if you have another service you'd like to use in-place of OpenAI, you can configure Mealie to use that instead, as long as it has an OpenAI-compatible API. For instance, a common self-hosted alternative to OpenAI is [Ollama](https://ollama.com/). To use Ollama with Mealie, change your `OPENAI_BASE_URL` to `http://localhost:11434/v1` (where `http://localhost:11434` is wherever you're hosting Ollama, and `/v1` enables the OpenAI-compatible endpoints). Note that you *must* provide an API key, even though it is ultimately ignored by Ollama.
|
||||
|
||||
If you wish to disable image recognition features (to save costs, or because your custom model doesn't support them) you can set `OPENAI_ENABLE_IMAGE_SERVICES` to `False`.
|
||||
If you wish to disable transcription features (to save costs, or because your custom model doesn't support them) you can set `OPENAI_ENABLE_TRANSCRIPTION_SERVICES` to `False`.
|
||||
|
||||
For more information on what configuration options are available, check out the [backend configuration](./backend-config.md#openai).
|
||||
|
||||
|
||||
|
||||
## OpenAI Features
|
||||
- The OpenAI Ingredient Parser can be used as an alternative to the NLP and Brute Force parsers. Simply choose the OpenAI parser while parsing ingredients (:octicons-tag-24: v1.7.0)
|
||||
- When importing a recipe via URL, if the default recipe scraper is unable to read the recipe data from a webpage, the webpage contents will be parsed by OpenAI (:octicons-tag-24: v1.9.0)
|
||||
- You can import an image of a written recipe, which is sent to OpenAI and imported into Mealie. The recipe can be hand-written or typed, as long as the text is in the picture. You can also optionally have OpenAI translate the recipe into your own language (:octicons-tag-24: v1.12.0)
|
||||
- You can import a recipe via a video URL (e.g., a YouTube link). The video is transcribed using OpenAI's Whisper model, and the transcription is parsed into a recipe (:octicons-tag-24: v3.13.0)
|
||||
@@ -76,7 +76,7 @@ nav:
|
||||
- Backend Configuration: "documentation/getting-started/installation/backend-config.md"
|
||||
- Security: "documentation/getting-started/installation/security.md"
|
||||
- Logs: "documentation/getting-started/installation/logs.md"
|
||||
- OpenAI: "documentation/getting-started/installation/open-ai.md"
|
||||
- AI Providers: "documentation/getting-started/installation/ai-providers.md"
|
||||
- Usage:
|
||||
- Backup and Restoring: "documentation/getting-started/usage/backups-and-restoring.md"
|
||||
- Permissions and Public Access: "documentation/getting-started/usage/permissions-and-public-access.md"
|
||||
|
||||
Reference in New Issue
Block a user