feat(backup): add user asset backup & migration module by leoleils · Pull Request #2457 · agentscope-ai/QwenPaw

leoleils · 2026-03-28T13:43:14Z

Description

Add a complete backup and migration system for CoPaw workspace assets. Users can export/import preferences, memories, skills, tools, and global config as portable ZIP packages, with daily auto-backup via cron, version compatibility checking, sensitive field redaction, and multi-agent batch support.

Related Issue: Relates to user requests for workspace backup and cross-device migration capability.

Security Considerations: Sensitive fields (api_key, bot_token, password, etc.) are auto-redacted on export. Existing sensitive values are preserved on import and not overwritten by redacted placeholders. ZIP path traversal prevention and size limits (500MB/1GB) are enforced. SHA256 checksum verification per asset.

Type of Change

Component(s) Affected

Checklist

I ran pre-commit run --all-files locally and it passes
If pre-commit auto-fixed files, I committed those changes and reran checks
I ran tests locally (pytest or as relevant) and they pass
Documentation updated (if needed)
Ready for review

Testing

# Export all assets for default agent
copaw assets export

# Export only specific types
copaw assets export --types skills,memories

# Export all agents
copaw assets export --all

# Import with overwrite strategy
copaw assets import <zip> --strategy overwrite

# Verify package integrity and compatibility
copaw assets verify <zip>

# Manual backup
copaw backup run
copaw backup run --all

# List and restore backups
copaw backup list
copaw backup restore latest

Add a complete backup and migration system for CoPaw workspace assets, enabling daily auto-backup, cross-device migration, and selective export/import of user data. New module: src/copaw/backup/ - models.py: data models (AssetType, AssetManifest, ExportOptions, etc.) - errors.py: InvalidAssetPackageError, IncompatibleVersionError, etc. - sanitizer.py: preference file sensitive field redaction - version_checker.py: version compatibility, strict validation, migration - exporter.py: asset export engine with concurrency-safe reads - importer.py: asset import engine with conflict resolution - scheduler.py: backup scheduler with retention policy CLI commands: - copaw assets export/import/verify - copaw backup run/list/restore - Support --all flag for multi-agent batch operations Integration: - src/copaw/app/crons/backup_job.py: cron job builder - src/copaw/app/workspace/service_factories.py: service registration - src/copaw/cli/main.py: lazy-loaded command registration Asset types: preferences, memories, skills, tools, global_config Security: - Sensitive fields auto-redacted on export, preserved on import - ZIP path traversal prevention, size limits (500MB/1GB) - SHA256 checksum verification per asset Tests: 93 tests (14 property-based + unit), all passing Pre-commit: flake8, mypy, black pass; pylint 9.83/10

github-actions · 2026-03-28T13:43:23Z

Welcome to CoPaw! 🐾

Hi @leoleils, this is your 8th Pull Request.

🙌 Join Developer Community

Thanks so much for your contribution! We'd love to invite you to join the official CoPaw developer group! You can find the Discord and DingTalk group links under the "Developer Community" section on our docs page:
https://copaw.agentscope.io/docs/community

We truly appreciate your enthusiasm—and look forward to your future contributions! 😊

We'll review your PR soon.

gemini-code-assist

Code Review

This pull request introduces a comprehensive backup and migration system for CoPaw, including engines for asset export and import, sensitive data sanitization, and version compatibility management. It also adds a backup scheduler with retention policies and new CLI commands for manual asset operations. The review feedback highlights opportunities to improve error visibility by logging tool collection failures, refine exception handling during manifest validation, and eliminate code duplication by centralizing shared utility functions. Additionally, it is recommended to use English for all error messages to maintain consistency across the codebase.

gemini-code-assist · 2026-03-28T13:45:59Z

src/copaw/backup/exporter.py

+        except (json.JSONDecodeError, UnicodeDecodeError):
+            pass


Silently ignoring JSON parsing errors when collecting tools can be confusing for users, as they might not realize why their tools were not backed up. It's better to log a warning to provide visibility into such issues.

Suggested change

except (json.JSONDecodeError, UnicodeDecodeError):

pass

except (json.JSONDecodeError, UnicodeDecodeError) as exc:

logger.warning("Failed to parse tools from %s, skipping tool collection: %s", agent_json, exc)

gemini-code-assist · 2026-03-28T13:45:59Z

src/copaw/backup/importer.py

+            except (json.JSONDecodeError, Exception) as exc:
+                raise InvalidAssetPackageError(
+                    f"Invalid manifest.json: {exc}",
+                ) from exc


Catching the broad Exception class can hide unexpected bugs and make debugging more difficult. When validating the manifest with Pydantic's model_validate, a pydantic.ValidationError is the expected exception on failure. Please catch more specific exceptions here. You may need to add from pydantic import ValidationError at the top of the file.

gemini-code-assist · 2026-03-28T13:45:59Z

src/copaw/backup/scheduler.py

+def _get_agent_id(workspace_dir: Path) -> str:
+    """Read agent id from workspace agent.json."""
+    import json
+
+    agent_json = workspace_dir / "agent.json"
+    if agent_json.exists():
+        try:
+            data = json.loads(agent_json.read_text(encoding="utf-8"))
+            return data.get("id", "unknown")
+        except (json.JSONDecodeError, OSError):
+            pass
+    return "unknown"


This _get_agent_id function is a duplicate of the one in src/copaw/backup/exporter.py. To follow the DRY (Don't Repeat Yourself) principle and improve maintainability, this function should be moved to a shared utility module (e.g., src/copaw/backup/utils.py) and imported in both scheduler.py and exporter.py.

gemini-code-assist · 2026-03-28T13:45:59Z

src/copaw/backup/version_checker.py

+    """
+    match = _VERSION_RE.match(version_str)
+    if not match:
+        raise ValueError(f"无效的版本格式: {version_str}")


This error message is in Chinese, which is inconsistent with the rest of the codebase (docstrings, comments, other logs, and CLI help text are in English). For consistency and to make the code more accessible to a wider range of contributors, it's recommended to use English for all code-level messages. This applies to other Chinese-language strings in this module and in importer.py as well. If internationalization is a goal, using a dedicated i18n library would be a better approach.

Suggested change

raise ValueError(f"无效的版本格式: {version_str}")

raise ValueError(f"Invalid version format: {version_str}")

gemini-code-assist · 2026-03-28T13:45:59Z

src/qwenpaw/cli/backup_cmd.py

+def _get_workspace_dir(agent_id: str = "default") -> Path:
+    """Resolve workspace directory for the given agent."""
+    try:
+        config = load_config()
+        if agent_id in config.agents.profiles:
+            ref = config.agents.profiles[agent_id]
+            return Path(ref.workspace_dir).expanduser()
+    except Exception:
+        pass
+    return WORKING_DIR
+
+
+def _get_all_agent_ids() -> list[str]:
+    """Return all configured agent IDs."""
+    try:
+        config = load_config()
+        return list(config.agents.profiles.keys())
+    except Exception:
+        return ["default"]
+


The helper functions _get_workspace_dir and _get_all_agent_ids are duplicated from src/copaw/cli/assets_cmd.py. To avoid code duplication and improve maintainability, these functions should be extracted into a shared utility module within the cli package, such as src/copaw/cli/utils.py, and then imported where needed.

- Remove unused variable in test_sanitizer_properties.py - Remove unused import (Any) in test_scheduler.py - Remove unused variable (export_result) in test_scheduler.py - Simplify empty list assertion in test_scheduler.py - Remove unused imports (_MIGRATIONS, get_migration_path) in test_version_checker_properties.py

- Remove unused imports (json, ConflictInfo, ExportResult) in test_models - Simplify empty list assertions to use implicit booleaness - Fix unused arguments in MockMemoryManager (test_integration) - Refactor roundtrip test to reduce statement count (R0915) - Extract _get_agent_id to shared backup/utils.py (DRY) - Replace broad Exception catch with ValidationError in importer - Add warning log for tool collection parse failures in exporter - Use English for error messages in version_checker - Extract CLI helpers to avoid duplication between assets/backup cmds

- Remove unused InsufficientStorageError import (test_importer.py) - Remove unused result variable (test_importer.py) - Remove unused HealthCheck import (test_importer_properties.py) - Add pylint disable for intentional protected-access in tests - Rename unused conflicts variable to _conflicts

- Fix E1133 non-iterable in assets_cmd.py (extract types_list) - Add 'from exc' to all raise SystemExit(1) in CLI commands - Remove unused imports in test_exporter.py - Add pylint disable for unused-argument in FakeMemoryManager - Prefix unused result variable with underscore in test_exporter - Add pylint disable for protected-access in test_importer.py

- Fix F821: add missing ExportResult import in integration tests - Fix E501: break all long lines to ≤79 chars across test files - Fix F841: remove unused variable in test_exporter - Fix C0411: correct import order (pydantic before copaw) in importer - Fix E1133: use types_list pattern for non-iterable in assets_cmd - Fix W0613: add pylint disable for unused args in test mocks - Fix W0212: add pylint disable for protected-access in tests - Run black + add-trailing-comma to match CI formatting

- Move pylint disable comments to the def line (not return type) for too-many-branches and too-many-statements - Move protected-access disable to the access line in service_factories - All pre-commit checks now pass locally: black, flake8, mypy, pylint

# Conflicts: # src/copaw/cli/main.py

- Core backup module: models, sanitizer, version checker, exporter, importer, scheduler - Backup cron job registration - CLI commands: copaw assets export/import/verify, copaw backup run/list/restore - REST API router: /backup/export, /import, /list, /restore, /config - Backup service factory registered in workspace - Console backup page with auto-backup settings, history, export/import - Agent selector for export with select-all support - i18n translations for en, zh, ja, ru - 94 unit + property-based tests (hypothesis)

…rt, export result with download button

…port success message

github-project-automation bot added this to QwenPaw Mar 28, 2026

github-project-automation bot moved this to Todo in QwenPaw Mar 28, 2026

leoleils requested a deployment to maintainer-approved March 28, 2026 13:43 — with GitHub Actions Waiting

leoleils mentioned this pull request Mar 28, 2026

[Feature]: User Asset Backup & Migration #2443

Open

gemini-code-assist bot reviewed Mar 28, 2026

View reviewed changes

leoleils requested a deployment to maintainer-approved March 28, 2026 13:52 — with GitHub Actions Waiting

leoleils requested a deployment to maintainer-approved March 28, 2026 14:02 — with GitHub Actions Waiting

leoleils requested a deployment to maintainer-approved March 28, 2026 14:37 — with GitHub Actions Waiting

leoleils requested a deployment to maintainer-approved March 28, 2026 14:46 — with GitHub Actions Waiting

leoleils requested a deployment to maintainer-approved March 28, 2026 15:09 — with GitHub Actions Waiting

leoleils requested a deployment to maintainer-approved March 28, 2026 15:26 — with GitHub Actions Waiting

github-actions bot mentioned this pull request Mar 29, 2026

🦞 OpenClaw 生态日报 2026-03-29 gsscsd/big_model_radar#114

Open

xieyxclack linked an issue Mar 29, 2026 that may be closed by this pull request

[Feature]: User Asset Backup & Migration #2443

Open

镭屿 added 4 commits April 8, 2026 22:49

Merge branch 'main' into feat/user-asset-backup-migration

d5e0d42

Merge branch 'main' into feat/user-asset-backup-migration

375427d

# Conflicts: # src/copaw/cli/main.py

merge: resolve conflict with main (lazy loading + backup route)

0ea9c63

leoleils requested a deployment to maintainer-approved April 11, 2026 00:24 — with GitHub Actions Waiting

fix: resolve lint issues - raise-missing-from and add validate_package

3c991f6

leoleils requested a deployment to maintainer-approved April 11, 2026 00:32 — with GitHub Actions Waiting

style: run prettier on backup frontend files

4df4d51

leoleils requested a deployment to maintainer-approved April 11, 2026 00:35 — with GitHub Actions Waiting

style: fix black formatting and flake8 line-too-long

4a31a67

leoleils requested a deployment to maintainer-approved April 11, 2026 00:43 — with GitHub Actions Waiting

github-actions bot mentioned this pull request Apr 11, 2026

🦞 OpenClaw 生态日报 2026-04-11 gsscsd/big_model_radar#168

Open

style: fix black+flake8 conflict in version_checker.py

a89b174

leoleils requested a deployment to maintainer-approved April 11, 2026 01:16 — with GitHub Actions Waiting

style: fix add-trailing-comma + black formatting

ea48d63

leoleils requested a deployment to maintainer-approved April 11, 2026 01:25 — with GitHub Actions Waiting

feat: add export download endpoint + browser auto-download

191dac2

leoleils requested a deployment to maintainer-approved April 11, 2026 01:56 — with GitHub Actions Waiting

fix: use agent workspace instead of WORKING_DIR, add file upload impo…

65cb8de

…rt, export result with download button

leoleils requested a deployment to maintainer-approved April 11, 2026 04:15 — with GitHub Actions Waiting

feat: complete i18n for all 4 locales, add strategy hints, improve im…

102b2eb

…port success message

leoleils requested a deployment to maintainer-approved April 11, 2026 07:56 — with GitHub Actions Waiting

style: prettier fix backup.ts and index.tsx

73056ab

leoleils requested a deployment to maintainer-approved April 11, 2026 08:00 — with GitHub Actions Waiting

github-actions bot mentioned this pull request Apr 12, 2026

🦞 OpenClaw 生态日报 2026-04-12 gsscsd/big_model_radar#173

Open

leoleils requested a deployment to maintainer-approved April 13, 2026 12:15 — with GitHub Actions Waiting

merge: resolve conflicts with main (copaw -> qwenpaw rename)

47e0a88

leoleils force-pushed the feat/user-asset-backup-migration branch from e7737c5 to 47e0a88 Compare April 13, 2026 14:17

leoleils requested a deployment to maintainer-approved April 13, 2026 14:17 — with GitHub Actions Waiting

github-actions bot mentioned this pull request Apr 14, 2026

🦞 OpenClaw 生态日报 2026-04-14 gsscsd/big_model_radar#184

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(backup): add user asset backup & migration module#2457

feat(backup): add user asset backup & migration module#2457
leoleils wants to merge 21 commits intoagentscope-ai:mainfrom
leoleils:feat/user-asset-backup-migration

leoleils commented Mar 28, 2026

Uh oh!

github-actions bot commented Mar 28, 2026

Welcome to CoPaw! 🐾

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Mar 28, 2026

Uh oh!

gemini-code-assist bot Mar 28, 2026

Uh oh!

gemini-code-assist bot Mar 28, 2026

Uh oh!

gemini-code-assist bot Mar 28, 2026

Uh oh!

gemini-code-assist bot Mar 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

-        except (json.JSONDecodeError, UnicodeDecodeError):
-            pass
+        except (json.JSONDecodeError, UnicodeDecodeError) as exc:
+            logger.warning("Failed to parse tools from %s, skipping tool collection: %s", agent_json, exc)

	raise ValueError(f"无效的版本格式: {version_str}")
	raise ValueError(f"Invalid version format: {version_str}")

Conversation

leoleils commented Mar 28, 2026

Description

Type of Change

Component(s) Affected

Checklist

Testing

Uh oh!

github-actions bot commented Mar 28, 2026

Welcome to CoPaw! 🐾

🙌 Join Developer Community

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Mar 28, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant