First tests with PhotoMaker + idea of installation for Automatic1111 users
I said it in a previous video: Nowadays, one of the biggest challenges of AI for content creation is to be able to generate stable characters and not just a serie of non-resembling chimeras.
So when I saw a video about a tool, PhotoMaker, that is able to use the characteristics of a picture to generate other pictures based on the characteristics of the first one, I wanted to test it.
The YouTube video that I list above, wants to compare it with IP adapter (Image Prompt Adapter), a controlnet.
How much is this tool better ?
Let's see that...
Installation of Photomaker for Automatic1111 users in a Windows system
The normal procedure to install PhotoMaker is described here for Windows.
As you can see, it is another Conda procedure, which is fine, the problem is the time and the space they take to install.
Here i don't try to install PhotoMaker for the Automatic1111 interface, I reuse the environment that I already have for Automatic1111 and use it to install PhotoMaker.
This is supposed to save time and space on your harddrive.
A little warning:
Here i explain how to install PhotoMaker with a non official method. If I use the software like that, it creates an error with "pip". If after that you have difficulties to run Automatic1111, use a command line window and use the script "activate.bat" that i mention below and then in the same terminal, start Automatic1111 by running the command line "webui.bat". If you have problem with pip, try: python3 -m ensurepip or python -m ensurepip if needed, as administrator. It fixes the problem, until you start PhotoMaker again.
Should you install PhotoMaker ?
To make it short, i would go here and find one of the online demo like here or there and then if you like it, install it.
If you don't want to play with your files in a way that can mess up another installation, don't do like me.
If you expect a version of PhotoMaker for automatic1111, if you want to save time or for whatever other reason you could think about, try this:
1-: Make sure you have Automatic1111 installed on your computer
Then check that you have the directory: "stable-diffusion-webui\venv\Scripts"
Make sure there is the file: "activate.bat" in this directory.
-2: Run CMD as administrator
(search for "CMD" in the start menu (right click), once you find it: right click and run as administrator.)
Then go to the directory given above: "stable-diffusion-webui\venv\Scripts"
In the location where Automatic1111 is installed.
Run the file activate.bat
(note that in the future, you will have to activate the environment first, before you start PhotoMaker. A custom bat file to run both bat files, could save you time.)
-3 Installation:
Go to the directory where you want to install PhotoMaker, then type:
git clone https://github.com/bmaltais/PhotoMaker.git cd PhotoMaker
(You are supposed to have Cuda and Pytorch with Automatic1111 already installed).
pip install -r requirements-windows.txt
Here you may see several errors or conflicts, but it worked anyway on my system.
(Here I add that I don't think I had to download or install any model...)
-4 Run the software
Type: gui.bat
And it is supposed to start the web interface.
-5 Testing:
Go to http://127.0.0.1:7860/ when the WebUi is started.
PhotoMaker comes with several examples, the famous Lenna, Isaac Newton, Scarlett Johanson and a Chinese actress that is called Yang Mi.
Sadly I don't know what Isaac Newton would really look like, for Lenna, she is famous for one picture, and I can't assess if Yang Mi looks like usual Yang Mi. So i only have one usable example: Scarlett Johanson...
Result:
I find the result quite convincing with her...
Check the picture to read my prompt, and don't forget the tag "img" with PhotoMaker.
But if I test with the provided examples, it is because I don't like so much the result that I get when I try with other people:
I think that my George Bush looks like John Kerry...
But otherwise, I like the picture.
Let's compare PhotoMaker with a controlNet "Ip adapter"...
My goal is to create the picture of a man with the face of George Bush running with or after a cat in Anime style.
I shouldn't use the name "George Bush" in the prompt and the only tool that I should use to make the picture look like George W Bush is the "IP adapter" with this picture. (Please have a look at the controlnet menu).
I had to complete my installation of ControlNet by adding in the directory: stable-diffusion-webui\extensions\sd-webui-controlnet\models
The files: ip-adapter_sd15.pth and ip-adapter_sd15_plus.pth
I start my generation like this:
man running with a grey cat running with him, art by Pixar, colorful
Negative prompt: easynegative
Steps: 30, Sampler: DPM++ SDE Karras, CFG scale: 7, Seed: 3276749894, Size: 512x512, Model hash: 8eadb35f02, Model: comicmixNSFWBy_v25, ControlNet 0: "Module: ip-adapter_clip_sd15, Model: ip-adapter_sd15 [6a3f6166], Weight: 1, Resize Mode: Crop and Resize, Low Vram: False, Processor Res: 512, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: Balanced, Hr Option: Both, Save Detected Map: True", TI hashes: "easynegative: 66a7279a88dd", Version: v1.7.0
I don't suggest to run it with "Hi res fix" because it would take a while.
This picture was easy to generate and besides the 2 tails of the cat, I think the picture of G Bush looks pretty good in Anime style.
At this point, I have to specify that after the first picture, i can't do much with PhotoMaker alone... but with Automatic1111, it is not the end.
With this StableDiffusion interface I can import my picture to the Img2Img panel and try to enhance the resolution:
I use a second Ip adapter and I put a resolution of 1024x1024:
man running with a grey cat running with him, art by Pixar, colorful
Negative prompt: easynegative
Steps: 30, Sampler: Euler a, CFG scale: 7, Seed: 3148416923, Size: 1024x1024, Model hash: 8eadb35f02, Model: comicmixNSFWBy_v25, Denoising strength: 0.43, ControlNet 0: "Module: ip-adapter_clip_sd15, Model: ip-adapter_sd15 [6a3f6166], Weight: 1, Resize Mode: Crop and Resize, Low Vram: False, Processor Res: 512, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: Balanced, Hr Option: Both, Save Detected Map: True", TI hashes: "easynegative: 66a7279a88dd", Version: v1.7.0
The cat has a problem and I can't fix its tail easily, so I edit it in "img2img inpaint" and I edit it out with the prompt "grey cat".
At the end, I have this:
I like the result. The cat could be a bit better.
Conclusion:
I don't want to minimize the value of PhotoMaker and maybe I didn't use the best settings, while I already had an experience with IP adapters (not with A1111). But for the moment, i like the use of "IP adapter" better.
Other prompt with ControlNet Ip adapter:
Here I use a resolution of 1024x1024 but i didn't check "hires fix".
Again, the cats could be a bit better.
One foot should be more visible.
So for the moment, i like Ip adapter better, even if it produces pictures that need to be adjusted.