Skip to content

Move blocked_status_codes from Session to the crawler #2139

Description

@vdusek

Blocked status codes are configured per Session (src/crawlee/sessions/_session.py), even though deciding whether a response means "blocked" is the crawler's concern - and the crawler already owns additional_http_error_status_codes and ignore_http_error_status_codes. Move it to a crawler option and keep the handling internal to the crawler.

JS implementation: apify/crawlee#3496

✍️ Drafted by Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    debtCode quality improvement or decrease of technical debt.t-toolingIssues with this label are in the ownership of the tooling team.

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions