Metadata-Version: 2.4
Name: cs-bs4utils
Version: 20260912
Summary: Various utility functions and classes for working with the HTML soup from `beautifulsoup4`.
Keywords: python3
Author-email: Cameron Simpson <cs@cskk.id.au>
Description-Content-Type: text/markdown
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Text Processing
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
Requires-Dist: beautifulsoup4
Requires-Dist: cs.gimmicks>=20260311
Requires-Dist: cs.lex>=20260912
Requires-Dist: cs.pfx>=20260912
Requires-Dist: icontract
Requires-Dist: lxml
Requires-Dist: typguard
Project-URL: MonoRepo Commits, https://bitbucket.org/cameron_simpson/css/commits/branch/main
Project-URL: Monorepo Git Mirror, https://github.com/cameron-simpson/css
Project-URL: Monorepo Hg/Mercurial Mirror, https://hg.sr.ht/~cameron-simpson/css
Project-URL: Source, https://github.com/cameron-simpson/css/blob/main/lib/python/cs/bs4utils.py

Various utility functions and classes for working with the HTML soup from `beautifulsoup4`.

*Latest release 20260912*:
Initial PyPI release.



Short summary:


* `as_xml`: Transform `tag` into an `lxml` XML element.


* `child_tags`: A generator yielding the immediate child tags of `child` whose tag name is `child_name`. If `child_name` is `None`, yield all the immediate child tags, skipping things like strings and comments.


* `find_heading`: Find the nearest heading satisfying the test `filter(tag)`. Return the tag or `None` is one is not found. The default `filter` tests that the heading is not empty. This uses `find_up` to locate the tag.


* `find_up`: A generator yielding `(found,ref)` 2-tuples obtained by search the tag tree left and up from `tag` using `.previous_sibling` and `.parent`, matching tags where `test(found)` is true.


* `printt_soup`: Print the contents of the soup via `cs.lex.printt` using `tabulate_soup` to make the table.


* `Table`: A `Widget` subclass representing an HTML TABLE tag.


* `tabulate_soup`: Return a table describing `soup` for use with `cs.lex.printt`. Connect tags with their child tags using Unicode box characters.


* `Widget`: Base class for various "widget" HTML constructs, such as a TABLE, or in principle anything else regular on a page.

# Functions

## as_xml(tag: bs4.element.Tag, *, E=None)

Transform `tag` into an `lxml` XML element.

## child_tags(tag, child_name: str | None = None) -> Iterable[bs4.element.Tag]

A generator yielding the immediate child tags of `child`
whose tag name is `child_name`.
If `child_name` is `None`, yield all the immediate child tags,
skipping things like strings and comments.

## find_heading(tag, filter: Callable[[bs4.element.Tag], bool] = <function <lambda> at 0x110363ec0>) -> bs4.element.Tag | None

Find the nearest heading satisfying the test `filter(tag)`.
Return the tag or `None` is one is not found.
The default `filter` tests that the heading is not empty.
This uses `find_up` to locate the tag.

## find_up(tag, test: Union[str, Callable[[bs4.element.Tag], bool]], *, first=False) -> Generator[tuple[bs4.element.Tag, bs4.element.Tag], tuple[None, None], NoneType]

A generator yielding `(found,ref)` 2-tuples obtained by
search the tag tree left and up from `tag` using `.previous_sibling`
and `.parent`, matching tags where `test(found)` is true.

The `test` may be a tag `.name` value (a string) or a callable
to evaluate a ound tag.

If `first` is true (default `False`) then the search stops
after the first match. If there are no matches the tuple
`(None,None)` is returned (this does not happen if `first`
is false).

A primary use case for this is to find the heading tag for `tag`.

In the tuple, `found` is the matched tag. `ref` is the reference
tag, the later sibling of `found` where the search started
for that level; if `found` is at the same level as `tag` then
`ref` will be `tag`.

For example, to locate the level 2 heading governing a tag:

    (h2,_), *_ = find_up(tag,lambda found: found.name == 'h2')

or more concisely:

    (h2,_), *_ = find_up(tag, 'h2')

or even:

    (h2,_), = find_up(tag, 'h2',first=True)

Note that the first two will cause Python to raise an exception
if there are no matches, while the third will provide `h2`
as `None`.

## printt_soup(tag: bs4.element.Tag, **printt_kw)

Print the contents of the soup via `cs.lex.printt`
using `tabulate_soup` to make the table.

## tabulate_soup(tag: bs4.element.Tag | bs4.element.NavigableString) -> list[list[str, str] | tuple]

Return a table describing `soup` for use with `cs.lex.printt`.
Connect tags with their child tags using Unicode box characters.

# Classes

## class Table(Widget)

A `Widget` subclass representing an HTML TABLE tag.

### Table.__init__(self, tag)

Scan the TABLE for the basic structures, used for the other properties etc later.

Note that if there was no TBODY, the immediate rows of the
TABLE are presented as though they were in a single TBODY.

### Table.IndexedCellValueType

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple.
If iterable is specified the tuple is initialized from iterable's items.

If the argument is a tuple, the return value is the same object.

### Table.__firstlineno__

int([x]) -> integer
int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments
are given.  If x is a number, return x.__int__().  For floating-point
numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string,
bytes, or bytearray instance representing an integer literal in the
given base.  The literal can be preceded by '+' or '-' and be surrounded
by whitespace.  The base defaults to 10.  Valid bases are 0 and 2-36.
Base 0 means to interpret the base from the string as an integer literal.
>>> int('0b100', base=0)
4

### Table.__static_attributes__

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple.
If iterable is specified the tuple is initialized from iterable's items.

If the argument is a tuple, the return value is the same object.

### Table.all_rows

Return all the rows from the header, bodies, and footer.

### Table.as_indexed_values(self, *, convert: Optional[Callable[[str, int, int, int, bs4.element.Tag], Any]] = None, convert_head_cell: Optional[Callable[[str, int, int, int, bs4.element.Tag], Any]] = None, convert_body_cell: Optional[Callable[[str, int, int, int, bs4.element.Tag], Any]] = None, convert_foot_cell: Optional[Callable[[str, int, int, int, bs4.element.Tag], Any]] = None, omit_header=False, omit_footer=False) -> list[list[tuple[str, int, int, int, bs4.element.Tag, typing.Any]]]

Return the table contents as a list-of-lists of indexed cell values.
Each inner list contains the cell value records from a row.

this is an elaborate counterpart to the `as_lists` method.

Parameters:
* `convert`: the default cell conversion function
* `convert_head`: the header cell conversion function, default from `convert`
* `convert_body`: the body cell conversion function, default from `convert`
* `convert_foot`: the footer cell conversion function, default from `convert`
* `omit_header`: do not include rows from the `THEAD` section
* `omit_footer`: do not include rows from the `TFOOT` section

The conversion functions accept the following positional parameters:
* `section_type`: one of `"THEAD"`, `"TBODY"` or `"TFOOT"`
* `section_index`: the index of the section, 0 for the
  header or footer but there may be multiple `TBODY` sections
* `row_index`: the index of the row within the section
* `col_index`: the index of the column within the row
* `cell`: the `TD` or `TH` tag for the cell
The function should return the converted value of `cell`.
The default conversion function returns `cell`.

The row and column indices supplied to the conversion unction
are of the _resolved_ cells, after expansion via the `colspan`
or `rowspan` values.
For example, a row with 3 cells whose second cell had a
`colspan=2` would be a list of 4 cells, with the second
original cell referenced in the second and third items of
the list; it _will_ be the same tag instance.

Each cell instance is converted only once; the same cell
spanning multiple columns or rows will have the same value
instance in the result record.

The resulting list-of-lists contains value records, a 6-tuple
of `(section_type,section_index,row_index,col_index,cell,value)`.
Note that the `row_index` and `col_index` are those of the
top left index where the `cell` was first encountered for
cells spanning multiple columns or rows.

Examples:

Convert every numeric cell to its `float` value, leave other cells as their text.

    def as_float(section_type, section_index, row_index, column_index, cell):
        text = cell.get_text.strip()
        try:
            value = float(text)
        except ValueError:
            value = text
        return value

    values = T.as_indexed_values(convert=as_float)

Convert only the body cells, keep the headers as tags, omit the footer:

    values = T.as_indexed_values(convert_body_cell=as_float, omit_footer=True)

### Table.as_lists(self, *, omit_header=False, omit_footer=False) -> list[list[bs4.element.Tag]]

Return the table contents as a list-of-lists-of-tags;
each inner list is a row of tags.
The innermost elements are the TH or TD tags.
Note that cells spanning multiple columns or rows via their
`colspan` or `rowspan` are the same reference.

Parameters:
* `omit_header`: do not include rows from the `THEAD` section
* `omit_footer`: do not include rows from the `TFOOT` section

### Table.body_rows

The rows from the table TBODY tags, if any.
Note that if there was no TBODY, the immediate rows of the
TABLE are presented as though they were in a single TBODY.

### Table.cell_colspan(cell: bs4.element.Tag) -> int

Compute the `colspan` value for a table cell.

### Table.cell_rowspan(cell: bs4.element.Tag) -> int

Compute the `rowspan` value for a table cell.

### Table.foot_rows

The rows from the table TFOOT, if any.

### Table.head_rows

The rows from the table THEAD, if any.

### Table.printt(self)

Print the table text.

### Table.row_cells(tr: bs4.element.Tag) -> list[bs4.element.Tag]

Return a list of the cells (`TD` or `TH`) from a `TR` tag.
`colspan` is supported by referencing the same cell multiple times.
Only `TD` and `TH` tags which are immediate children of the `TR` are recognised.

### Table.section_rows(section: bs4.element.Tag | None) -> list[list[bs4.element.Tag]]

Return the rows from a table section such as `THEAD`, `TBODY`, or `TFOOT`.
`rowspan` is supported by referencing the same cell in lower rows.

### Table.title

The title of the table, from the caption or the nearest heading.

## class Widget

Base class for various "widget" HTML constructs, such as a
TABLE, or in principle anything else regular on a page.

A `Widgwt` supplies:
- `__init__(tag)` to record the target BS4 tag, typically the
  top level tag encompassing the wudget
- `find_all(soup)`: returning a list of the top level tags
  within the BS4 tag `soup`; the default method calls
  `soup.find_all()` with the lower case version of the class
  name via `soup.find_all()`
- `scan(soup)`: a factory method calling `cls(tag)` for every
  tag found by `find_all(soup)`

Everything else in a subclass supports whatever needs doing
with the widget; the `Table` class is an exemplar:
- its `__init__` method passes the tag to `super().__init__()`
  as normal, then find s a few top level things about the table
  - the caption, header, bodies, footer
- the default `find_all` is used because the lass name matches
  the HTML tag name
- everything else more complex is provided as methods or
  `@cached_property` properties, computed on demand

### Widget.__init__(self, tag: bs4.element.Tag)

Initialise this `Widget` by saving `tag` as `self.tag` and
then calling `self.scan()`.

### Widget.__dict__

Read-only proxy of a mapping.

### Widget.__firstlineno__

int([x]) -> integer
int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments
are given.  If x is a number, return x.__int__().  For floating-point
numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string,
bytes, or bytearray instance representing an integer literal in the
given base.  The literal can be preceded by '+' or '-' and be surrounded
by whitespace.  The base defaults to 10.  Valid bases are 0 and 2-36.
Base 0 means to interpret the base from the string as an integer literal.
>>> int('0b100', base=0)
4

### Widget.__static_attributes__

Built-in immutable sequence.

If no argument is given, the constructor returns an empty tuple.
If iterable is specified the tuple is initialized from iterable's items.

If the argument is a tuple, the return value is the same object.

### Widget.find_all(soup) -> list[bs4.element.Tag]

The default `find_all` finds tags from `soup` whose name matches the class name.

### Widget.scan(soup) -> list[typing.Self]

Return a list of all `Widget`s of this type found in `soup`.

# Release Log



*Release 20260912*:
Initial PyPI release.
