diff --git a/README.md b/README.md index 4c53df6..8358749 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,7 @@ This work presents Depth Anything, a highly practical solution for robust monocu ## News +* **2024-02-05:** [Depth Anything Gallery](./gallery.md) is released. Thank all the users! * **2024-02-02:** Depth Anything serves as the default depth processor for [InstantID](https://github.com/InstantID/InstantID) and [InvokeAI](https://github.com/invoke-ai/InvokeAI/releases/tag/v3.6.1). * **2024-01-25:** Support [video depth visualization](./run_video.py). * **2024-01-23:** The new ControlNet based on Depth Anything is integrated into [ControlNet WebUI](https://github.com/Mikubill/sd-webui-controlnet) and [ComfyUI's ControlNet](https://github.com/Fannovel16/comfyui_controlnet_aux). diff --git a/assets/gallery/3d_video.gif b/assets/gallery/3d_video.gif new file mode 100644 index 0000000..ad43495 Binary files /dev/null and b/assets/gallery/3d_video.gif differ diff --git a/assets/gallery/3d_vis1.gif b/assets/gallery/3d_vis1.gif new file mode 100644 index 0000000..63cfeab Binary files /dev/null and b/assets/gallery/3d_vis1.gif differ diff --git a/assets/gallery/3d_vis2.gif b/assets/gallery/3d_vis2.gif new file mode 100644 index 0000000..5682437 Binary files /dev/null and b/assets/gallery/3d_vis2.gif differ diff --git a/assets/gallery/aerial_image.jpg b/assets/gallery/aerial_image.jpg new file mode 100644 index 0000000..c12cb6c Binary files /dev/null and b/assets/gallery/aerial_image.jpg differ diff --git a/assets/gallery/animation_image.jpg b/assets/gallery/animation_image.jpg new file mode 100644 index 0000000..86b3af7 Binary files /dev/null and b/assets/gallery/animation_image.jpg differ diff --git a/assets/gallery/artistic_image.jpg b/assets/gallery/artistic_image.jpg new file mode 100644 index 0000000..1f45f0a Binary files /dev/null and b/assets/gallery/artistic_image.jpg differ diff --git a/assets/gallery/astronaut_cat.jpg b/assets/gallery/astronaut_cat.jpg new file mode 100644 index 0000000..655eb4d Binary files /dev/null and b/assets/gallery/astronaut_cat.jpg differ diff --git a/assets/gallery/building.gif b/assets/gallery/building.gif new file mode 100644 index 0000000..26f822f Binary files /dev/null and b/assets/gallery/building.gif differ diff --git a/assets/gallery/cat.jpg b/assets/gallery/cat.jpg new file mode 100644 index 0000000..f2f51a4 Binary files /dev/null and b/assets/gallery/cat.jpg differ diff --git a/assets/gallery/chibi_cateared_character.jpg b/assets/gallery/chibi_cateared_character.jpg new file mode 100644 index 0000000..190270f Binary files /dev/null and b/assets/gallery/chibi_cateared_character.jpg differ diff --git a/assets/gallery/cyber_rabbit.jpg b/assets/gallery/cyber_rabbit.jpg new file mode 100644 index 0000000..6af1fa5 Binary files /dev/null and b/assets/gallery/cyber_rabbit.jpg differ diff --git a/assets/gallery/dalle_bear.jpg b/assets/gallery/dalle_bear.jpg new file mode 100644 index 0000000..b2911e0 Binary files /dev/null and b/assets/gallery/dalle_bear.jpg differ diff --git a/assets/gallery/diner_scene.jpg b/assets/gallery/diner_scene.jpg new file mode 100644 index 0000000..057ebc3 Binary files /dev/null and b/assets/gallery/diner_scene.jpg differ diff --git a/assets/gallery/distracted_man.jpg b/assets/gallery/distracted_man.jpg new file mode 100644 index 0000000..653d085 Binary files /dev/null and b/assets/gallery/distracted_man.jpg differ diff --git a/assets/gallery/elon_musk.jpg b/assets/gallery/elon_musk.jpg new file mode 100644 index 0000000..fbc285e Binary files /dev/null and b/assets/gallery/elon_musk.jpg differ diff --git a/assets/gallery/eye-stalked.jpg b/assets/gallery/eye-stalked.jpg new file mode 100644 index 0000000..c824dbf Binary files /dev/null and b/assets/gallery/eye-stalked.jpg differ diff --git a/assets/gallery/football_game.jpg b/assets/gallery/football_game.jpg new file mode 100644 index 0000000..849b0ab Binary files /dev/null and b/assets/gallery/football_game.jpg differ diff --git a/assets/gallery/gamer_celebration.jpg b/assets/gallery/gamer_celebration.jpg new file mode 100644 index 0000000..dfbf35a Binary files /dev/null and b/assets/gallery/gamer_celebration.jpg differ diff --git a/assets/gallery/grilled_chicken_skewers.jpg b/assets/gallery/grilled_chicken_skewers.jpg new file mode 100644 index 0000000..5ce0bd6 Binary files /dev/null and b/assets/gallery/grilled_chicken_skewers.jpg differ diff --git a/assets/gallery/indoor_moving.gif b/assets/gallery/indoor_moving.gif new file mode 100644 index 0000000..09beef7 Binary files /dev/null and b/assets/gallery/indoor_moving.gif differ diff --git a/assets/gallery/iron_man.jpg b/assets/gallery/iron_man.jpg new file mode 100644 index 0000000..1da3ef3 Binary files /dev/null and b/assets/gallery/iron_man.jpg differ diff --git a/assets/gallery/minecraft.jpg b/assets/gallery/minecraft.jpg new file mode 100644 index 0000000..6888c2e Binary files /dev/null and b/assets/gallery/minecraft.jpg differ diff --git a/assets/gallery/monster.png b/assets/gallery/monster.png new file mode 100644 index 0000000..a751944 Binary files /dev/null and b/assets/gallery/monster.png differ diff --git a/assets/gallery/monument_valley.jpg b/assets/gallery/monument_valley.jpg new file mode 100644 index 0000000..c0d6def Binary files /dev/null and b/assets/gallery/monument_valley.jpg differ diff --git a/assets/gallery/ocean.jpg b/assets/gallery/ocean.jpg new file mode 100644 index 0000000..108c8ac Binary files /dev/null and b/assets/gallery/ocean.jpg differ diff --git a/assets/gallery/painted_tunnel.jpg b/assets/gallery/painted_tunnel.jpg new file mode 100644 index 0000000..652baab Binary files /dev/null and b/assets/gallery/painted_tunnel.jpg differ diff --git a/assets/gallery/racing_car.gif b/assets/gallery/racing_car.gif new file mode 100644 index 0000000..ce1f10f Binary files /dev/null and b/assets/gallery/racing_car.gif differ diff --git a/assets/gallery/raft_painting.jpg b/assets/gallery/raft_painting.jpg new file mode 100644 index 0000000..75b0756 Binary files /dev/null and b/assets/gallery/raft_painting.jpg differ diff --git a/assets/gallery/reconstruction.jpg b/assets/gallery/reconstruction.jpg new file mode 100644 index 0000000..c087878 Binary files /dev/null and b/assets/gallery/reconstruction.jpg differ diff --git a/assets/gallery/reconstruction2.jpeg b/assets/gallery/reconstruction2.jpeg new file mode 100644 index 0000000..3fb3747 Binary files /dev/null and b/assets/gallery/reconstruction2.jpeg differ diff --git a/assets/gallery/robotic_knight.jpg b/assets/gallery/robotic_knight.jpg new file mode 100644 index 0000000..2746406 Binary files /dev/null and b/assets/gallery/robotic_knight.jpg differ diff --git a/assets/gallery/skull.jpg b/assets/gallery/skull.jpg new file mode 100644 index 0000000..9f7cb77 Binary files /dev/null and b/assets/gallery/skull.jpg differ diff --git a/assets/gallery/surprised_bald_man.jpg b/assets/gallery/surprised_bald_man.jpg new file mode 100644 index 0000000..5958570 Binary files /dev/null and b/assets/gallery/surprised_bald_man.jpg differ diff --git a/assets/gallery/tearful_green_frog.jpg b/assets/gallery/tearful_green_frog.jpg new file mode 100644 index 0000000..fe1a8d2 Binary files /dev/null and b/assets/gallery/tearful_green_frog.jpg differ diff --git a/assets/gallery/videoframe_10000.png b/assets/gallery/videoframe_10000.png new file mode 100644 index 0000000..43505ed Binary files /dev/null and b/assets/gallery/videoframe_10000.png differ diff --git a/metric_depth/README.md b/metric_depth/README.md index 60dbb31..323d2a0 100644 --- a/metric_depth/README.md +++ b/metric_depth/README.md @@ -53,7 +53,7 @@ Please follow [ZoeDepth](https://github.com/isl-org/ZoeDepth) to prepare the tra ## Evaluation -Make sure you have downloaded our pre-trained models [here](https://huggingface.co/spaces/LiheYoung/Depth-Anything/tree/main/checkpoints_metric_depth) and put them under the ``checkpoints`` directory. +Make sure you have downloaded our pre-trained metric-depth models [here](https://huggingface.co/spaces/LiheYoung/Depth-Anything/tree/main/checkpoints_metric_depth) (for evaluation) and pre-trained relative-depth model [here](https://huggingface.co/spaces/LiheYoung/Depth-Anything/blob/main/checkpoints/depth_anything_vitl14.pth) (for initializing the encoder) and put them under the ``checkpoints`` directory. Indoor: ```bash @@ -67,7 +67,7 @@ python evaluate.py -m zoedepth --pretrained_resource="local::./checkpoints/depth ## Training -Please first download our Depth Anything pre-trained models [here](https://huggingface.co/spaces/LiheYoung/Depth-Anything/blob/main/checkpoints/depth_anything_vitl14.pth), and put it under the ``checkpoints`` directory. +Please first download our Depth Anything pre-trained model [here](https://huggingface.co/spaces/LiheYoung/Depth-Anything/blob/main/checkpoints/depth_anything_vitl14.pth), and put it under the ``checkpoints`` directory. ```bash python train_mono.py -m zoedepth -d --pretrained_resource="" diff --git a/semseg/README.md b/semseg/README.md index c474b71..8e26f38 100644 --- a/semseg/README.md +++ b/semseg/README.md @@ -37,18 +37,22 @@ Note that our results are obtained *without* Mapillary pre-training. - [Cityscapes-ViT-L-mIoU-86.4](https://huggingface.co/spaces/LiheYoung/Depth-Anything/blob/main/checkpoints_semseg/cityscapes_vitl_mIoU_86.4.pth) - [ADE20K-ViT-L-mIoU-59.4](https://huggingface.co/spaces/LiheYoung/Depth-Anything/blob/main/checkpoints_semseg/ade20k_vitl_mIoU_59.4.pth) -**Note:** If you want to reproduce the training process, please 1) download the [Depth Anything pre-trained model](https://huggingface.co/spaces/LiheYoung/Depth-Anything/blob/main/checkpoints/depth_anything_vitl14.pth) (to initialize the encoder) and 2) put it under the ``checkpoints`` folder. - ## Installation Please refer to [MMSegmentation](https://github.com/open-mmlab/mmsegmentation/blob/main/docs/en/get_started.md#installation) for instructions. +Please also install mmdet to support Mask2Former: +```bash +pip install "mmdet>=3.0.0rc4" +``` + After installation: - move our [config/depth_anything](./config/depth_anything/) to mmseg's [config](https://github.com/open-mmlab/mmsegmentation/tree/main/configs) - move our [dinov2.py](./dinov2.py) to mmseg's [backbones](https://github.com/open-mmlab/mmsegmentation/tree/main/mmseg/models/backbones) - add DINOv2 in mmseg's [models/backbones/\_\_init\_\_.py](https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/models/backbones/__init__.py) - download our provided [torchhub](https://github.com/LiheYoung/Depth-Anything/tree/main/torchhub) directory and put it at the root of your working directory -For training or inference with our pre-trained models, please refer to MMSegmentation [instructions](https://github.com/open-mmlab/mmsegmentation/blob/main/docs/en/user_guides/4_train_test.md). +**Note:** If you want to reproduce the **training** process, please 1) download the [Depth Anything pre-trained model](https://huggingface.co/spaces/LiheYoung/Depth-Anything/blob/main/checkpoints/depth_anything_vitl14.pth) (to initialize the encoder) and 2) put it under the ``checkpoints`` folder. +For training or inference with our pre-trained models, please refer to MMSegmentation [instructions](https://github.com/open-mmlab/mmsegmentation/blob/main/docs/en/user_guides/4_train_test.md).