tokenizertestresults.txt

来自「java操作excel的类」· 文本 代码 · 共 102 行

  ===============================================
  4                   | 4
  this                | this
  is                  | is
  a                   | a
  test                | test
  ===============================================
  0                   | 0
  ===============================================
  1                   | 1
  1                   | 1
  no delims           | no delims
  ===============================================
  1                   | 1
  string              | string
  ===============================================
  3                   | 3
  A                   | A
  string              | string
  B                   | B
  ===============================================
  someURL             | someURL
  ?name               | ?name
  value               | value
  name                | name
  value               | value
  ===============================================
  someURL             | someURL
  name                | name
  value               | value
  name                | name
  value               | value
  ===============================================
  9                   | 9
  (                   | (
  aaa                 | aaa
  *                   | *
  (                   | (
  b                   | b
  +                   | +
  c1                  | c1
  )                   | )
  )                   | )
  ===============================================
  5                   | 5
  one                 | one
  two                 | two
  four                | four
  five                | five
  eight               | eight
  ===============================================
  8                   | 8
  ===============================================
  9                   | 9
  ===============================================
  2                   | 2
  ===============================================
  1                   | 1
  ===============================================
  17                  | 17
  ===============================================
  one                 | one
  4                   | 4
  ===============================================
  list                | list
                      | 
  0                   | 0
  ===============================================
  list                | list
                      | 
  1                   | 1
  ===============================================
  list                | list
                      | 
  2                   | 2
  ===============================================
  this                | this
  is                  | is
  a                   | a
  test                | test
  ===============================================
  token               | token
  rest of string      | rest of string
  false               | false
  ===============================================
  4                   | 4
  testing             | testing
  4                   | 4
  testing             | testing
  3                   | 3
  the                 | the
  the                 | the
  the                 | the
  peek                | peek
  peek                | peek
  true                | true
  method              | method
  true                | true
  method              | method
  false               | false
  ===============================================