Thai Sentence Segmentation Demo

Tools used for Sentence Segmentation:

Paper about Sentence Segmentation, as part of a QA-System: see Section III., (A))
POS-Tagging using Orchid97-Corpus: link
Word Segmentation using LongLexTo: link - a rewritten implementation in Python is currently used for improved performance.
WebApp using Flask and mini.css (mini.css is being fetch by CDN jsDelivr)
Privacy notice

{% if request.method == 'POST' %} {% macro pos_tag(word, pos) -%} {{ word if pos not in ('SBS', 'NSBS') else " " | safe }} {%- endmacro %} {% macro pos_tag_seq(word_pos, type=None) -%} {% if type is not none %}


        {%- if type == 'frag' -%}
          Fragment
        {%- elif type == 'frag_sep' -%}
          Fragment separator (non sentence break space)
        {%- elif type == 'sent_sep' -%}
          Sentence separator (sentence break space)
        {%- elif type == 'frag_konj?' -%}
          Konjunction
        {%- else -%}
          ...
        {%- endif -%}{# if type == 'frag' #}

{% endif %}{# if type is not None #} {% for wp in word_pos -%} {{ pos_tag(wp[0], wp[1]) }} {# {{ pos_tag(wp[0], '{}_{}_{}'.format(wp[1], wp[2], wp[3])) }} #} {%- endfor %} {%- endmacro %}

{% if all_sentences %} {% for nr, paragraph, tree, sentences in all_sentences %} {% set is_other_row = loop.index % 2 == 0 %} {% if add_pos %} {% endif %}{# if add_pos #} {% if add_seg_tree %} {% endif %}{# if add_seg_tree #} {% if (sentences | length == 1) and (sentences[0].pos | length == paragraph.pos | length) %} {% else %}{# if (sentences | length == 1) and (sentences[0].pos | length == paragraph.pos | length) #} {% for sentence in sentences %} {% if add_pos %} {% endif %}{# if add_pos #} {% endfor %}{# for sentence in sentences #} {% endif %}{# if (sentences | length == 1) and (sentences[0].pos | length == paragraph.pos | length) #} {% endfor %}{# for nr, paragraph, tree, sentences in all_sentences #}

Sentences
Nr	Sentence
Input {{ nr }}	{{ paragraph \| string \| escape }}
POS {{ nr }}	{{ pos_tag_seq(paragraph.pos) }}
Segmentation {{ nr }}	{% for line in tree %} {% if line[0] == 'sep' %} Sentence Break: {% for subline in line[1] %} {{ pos_tag_seq(subline[1], subline[0]) }} {% endfor %}{# for subline in line[1] #} {% else %}{# if line[0] == 'sep' #} Sentence {{ loop.index // 2 + 1 }}: {% for subline in line[1] %} {{ pos_tag_seq(subline[1], subline[0]) }} {% endfor %}{# for subline in line[1] #} {% endif %}{# if line[0] == 'sep' #} {% endfor %}{# for line in tree #}
Result {{ nr }}	No segmentation possible/neccessary!
Segment {{ nr }} - {{ loop.index }}	{{ sentence \| string \| escape }}
POS	{{ pos_tag_seq(sentence.pos) }}

{% endif %}{# if all_sentences #} {% endif %}{# if request.method == 'POST' #}