主页 / coding / 问题 / 77850833
Accepted
danday74
Asked: 2024-01-20 19:30:38 +0800 CST

如何将字符串分解为单词并跟踪每个单词的索引（在原始字符串内）？

  • 772

我有一个字符串：

const str = 'a string, a long string'

我想将其分解为单词（这里没问题），然后跟踪原始字符串中每个单词的索引。

实际结果：

[
  { word: 'a',      idx: 0 },
  { word: 'string', idx: 2 },
  { word: 'a',      idx: 0 },
  { word: 'long',   idx: 12 },
  { word: 'string', idx: 2 }
]

期望的结果：

[
  { word: 'a',      idx: 0 },
  { word: 'string', idx: 2 },
  { word: 'a',      idx: 10 },
  { word: 'long',   idx: 12 },
  { word: 'string', idx: 17 }
]

到目前为止的代码：

const str = 'a string, a long string'

const segmenter = new Intl.Segmenter([], { granularity: 'word' })

const getWords = str => {
  const segments = segmenter.segment(str)
  return [...segments]
    .filter(s => s.isWordLike)
    .map(s => s.segment)
}

const words = getWords(str)

const result = words.map(word => ({
  word,
  idx: str.indexOf(word)
}))

console.log(result)

javascript

4 个回答

  1. Best Answer

    您正在迭代的对象包含 ，并且segment无论是否包含isWordLike，也包含index：

    const str = 'a string, a long string'

const segmenter = new Intl.Segmenter([], { granularity: 'word' })

const getWordsWithIndexes = str => {
  const segments = segmenter.segment(str)
  return [...segments]
    .filter(s => s.isWordLike)
    .map(s => ({ idx: s.index, word: s.segment }))
}

const result = getWordsWithIndexes(str)

console.log(result)

    这是类型定义：

    interface SegmentData {
    /** A string containing the segment extracted from the original input string. */
    segment: string;
    /** The code unit index in the original input string at which the segment begins. */
    index: number;
    /** The complete input string that was segmented. */
    input: string;
    /**
     * A boolean value only if granularity is "word"; otherwise, undefined.
     * If granularity is "word", then isWordLike is true when the segment is word-like (i.e., consists of letters/numbers/ideographs/etc.); otherwise, false.
     */
    isWordLike?: boolean;
}
    • 3

  2. String.matchAll也许是一个用于检索单词和索引的想法。

    或者只是减少Intl.Segmenter.segment结果。

    就像是：

    const str = 'a string, a long string';

console.log(getWordStartIndexes(str));
console.log(segment2Words(str));

function getWordStartIndexes(str) {
   return [...str.matchAll(/(\p{L}+?)([\p{Z}\p{P}]|$)/gu)]
    .map( match => ({word: match[1], index: match.index }) );
   
}

function segment2Words(str) {
  return [...new Intl.Segmenter([], { granularity: 'word' }).segment(str)]
    .reduce( (acc, {segment, index, isWordLike}) => 
      isWordLike ? [...acc, {word: segment, index}] : acc, []);
}
    .as-console-wrapper {
    max-height: 100% !important;
}

    • 1

  3. 您可以尝试使用正则表达式，它比分段器快 25 倍：

    const result = [];
'a string, a long string'.replace(/\w+/g, (word, idx) => result.push({word, idx}));

result.forEach(r => console.log(JSON.stringify(r)));

    ` Chrome/120
---------------------------------------------------------------
Alexander           1.00x  |  x1000000  234  239  240  245  246
cached Segmenter   26.79x  |   x100000  627  636  643  650  663
Segmenter          71.79x  |    x10000  168  169  170  171  177
---------------------------------------------------------------
https://github.com/silentmantra/benchmark `


    const segmenter = new Intl.Segmenter([], { granularity: 'word' })

// @benchmark Segmenter
{
const segmenter = new Intl.Segmenter([], { granularity: 'word' })

const getWordsWithIndexes = str => {
  const segments = segmenter.segment(str)
  return [...segments]
    .filter(s => s.isWordLike)
    .map(s => ({ idx: s.index, word: s.segment }))
}

getWordsWithIndexes('a string, a long string');
}

// @benchmark cached Segmenter
{
const getWordsWithIndexes = str => {
  const segments = segmenter.segment(str)
  return [...segments]
    .filter(s => s.isWordLike)
    .map(s => ({ idx: s.index, word: s.segment }))
}

getWordsWithIndexes('a string, a long string');
}

// @benchmark Alexander
const result = [];
'a string, a long string'.replace(/\w+/g, (word, idx) => result.push({word, idx}));
result;

/*@end*/eval(atob('e2xldCBlPWRvY3VtZW50LmJvZHkucXVlcnlTZWxlY3Rvcigic2NyaXB0Iik7aWYoIWUubWF0Y2hlcygiW2JlbmNobWFya10iKSl7bGV0IHQ9ZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgic2NyaXB0Iik7dC5zcmM9Imh0dHBzOi8vY2RuLmpzZGVsaXZyLm5ldC9naC9zaWxlbnRtYW50cmEvYmVuY2htYXJrL2xvYWRlci5qcyIsdC5kZWZlcj0hMCxkb2N1bWVudC5oZWFkLmFwcGVuZENoaWxkKHQpfX0='));

    • 1

  4. 我将您的字符串分解为包含单词和单词索引的对象数组。

    const str = 'a string, a long string';

const words = str.split(' ').map((word, index) => ({ word, index }));

console.log(words)

    如果您希望标点符号作为单词，您可以使用正则表达式。

    const words = str.split(/\s+|(?=\p{P})|(?<=\p{P})/u).map((word, index) => ({ word, index }));
    • -2

相关问题