Skip to content

Call tokenize with just a tokenizer#901

Merged
IsaacWarren merged 2 commits intomainfrom
isaac/tokenize_with_just_tokenizer
Oct 30, 2025
Merged

Call tokenize with just a tokenizer#901
IsaacWarren merged 2 commits intomainfrom
isaac/tokenize_with_just_tokenizer

Conversation

@IsaacWarren
Copy link
Contributor

Changes included in this PR

As title, allow calling series.ai.tokenize with just a tokenizer

Testing strategy

New test

User facing changes

Simpler UX in some cases.

Checklist

  • Pipelines passed before requesting review. To run CI you must include [run CI] in your commit message.
  • I am familiar with the Contributing Guide
  • I have installed + ran pre-commit hooks.

@codecov
Copy link

codecov bot commented Oct 29, 2025

Codecov Report

❌ Patch coverage is 71.42857% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.79%. Comparing base (c33fbb5) to head (84007bd).
⚠️ Report is 98 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #901      +/-   ##
==========================================
+ Coverage   66.68%   68.79%   +2.11%     
==========================================
  Files         186      194       +8     
  Lines       66795    67528     +733     
  Branches     9507     9592      +85     
==========================================
+ Hits        44543    46458    +1915     
+ Misses      19572    18239    -1333     
- Partials     2680     2831     +151     

Copy link
Collaborator

@DrTodd13 DrTodd13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. thanks

Copy link
Contributor

@scott-routledge2 scott-routledge2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@IsaacWarren IsaacWarren merged commit e5b6e97 into main Oct 30, 2025
27 checks passed
@IsaacWarren IsaacWarren deleted the isaac/tokenize_with_just_tokenizer branch October 30, 2025 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants