mirror of
https://github.com/kvcache-ai/Mooncake.git
synced 2026-11-05 01:17:55 +00:00
[GH-ISSUE #727] object_already_exists；mark_complete(): CHECK_EQ(status_, PROCESSING) #229
Labels
No labels
ascend
auto-closed
bug
bug
CI/Build
Common
documentation
enhancement
feature request
good first issue
help wanted
high priority
MetaX
nightly-failure
pull-request
PyTorch Backend
question
RFC
Roadmap
stale
Store
Tests
Transfer Engine
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
kvcache-ai/Mooncake#229
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @zx-ai on GitHub (Aug 8, 2025).
Original GitHub issue: https://github.com/kvcache-ai/Mooncake/issues/727
Before calling batchPut, I first call the batchIsExist() interface to find out which keys do not exist (returning -704). Only for those non-existent keys will I call batchPut to store them into mooncake_master. However, I still encounter the following error:
I20250808 16:04:00.720455 47968852977408 master_service.cpp:350] key=0000000000012437241962574060620, info=object_already_exists
I20250808 16:04:00.720670 47968859281152 master_service.cpp:350] key=0000000000012437241962574060620,info=object_already_exists
F20250808 16:04:00.721639 47968859281152 types.h:307] Check failed: status_ == ReplicaStatus::PR0CESSING (COMPLETE Vs. PROCESSING)
Why do I still get a "object_already_exists" error when I only put keys that don't exist (indicated by -704)?
Should I remove this CHECK_EQ restriction? Would storing duplicate keys cause the mooncake store to crash?
@ykwd commented on GitHub (Aug 11, 2025):
Thanks for the report!
If things happened exactly as you described, then this might indeed be a bug. However, we haven’t encountered such a case so far. Could you share the steps or a minimal example to help us reproduce the issue?
@xiaguan commented on GitHub (Aug 11, 2025):
Did you use the latest release? I think we removed it a while ago.
@zx-ai commented on GitHub (Aug 11, 2025):
@xiaguan I have used the latest release, when object_already_exists ,the mooncake_master do not crash.
@zx-ai commented on GitHub (Aug 11, 2025):
@ykwd This issue occurs intermittently. I cannot provide a minimal reproducible example. However, it can be observed that the issue is more likely to occur when numerous processes are simultaneously performing put, get, and batchIsExist operations. I suspect this might be due to network congestion during RPC communication, causing another process to push the same key between the time when the key's status is checked and when the current process attempts to push it.
@ykwd commented on GitHub (Aug 11, 2025):
@zx-ai I see. If multiple requests attempt to put the same key, any request after the first one may receive an
object_already_existserror. This is an expected behavior and should not cause any issues for upper-level applications.@github-actions[bot] commented on GitHub (Jun 27, 2026):
This issue has had no activity for 90 days and will be closed in 7 days if there is no further activity. Please comment or react if it should stay open.
@github-actions[bot] commented on GitHub (Jul 6, 2026):
Closing due to 3 months of inactivity. If this is still relevant, please comment and we can reopen.