Skip to content

Extended Memory Usage Optimizations #4703

@tarunKoyalwar

Description

@tarunKoyalwar

Please describe your feature request:

context #4680 . it was observed from testing that following changes need to be done to address allocations / peak memory usage

### Tasks
- [ ] Implement a immutable copy of bytes.buffer to avoid allocation when doing .String() and allow segmentation to consolidate body and full response into single byte array
- [ ] Remove Disk Backed Streaming & Host Iteration in hybridmap and use typedCache with batching (batch targets into 10-50k chunks while running in template-spray mode)
- [ ] Investigate need of deepcopier in fastdialer ( observed in cpu profiles)
- [ ] Experiment: client side caching of response body when response header contains `etag` or other known headers . this will reduce memory usage by reusing immutable response bodies ( when scanning a target which is serving 404 or content from s3 bucket etc)
- [ ] Using Javascript Runtime instead of projectdiscovery/dsl , design helper functions to accept both []byte slice or io.Reader to avoid unnecessary allocations
- [ ] Refactor to make Executers stateless and move state to scan.ScanContext

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type: EnhancementMost issues will probably ask for additions or changes.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions