Does the length of content in search results vary depending on the query language?

[JeongYunLee]

Hi, I'm building a langgraph using tavily and have a question.

When using tavily's basic search code in Python, I noticed that different sources of information are retrieved depending on the language of the query, even if the question is the same. When I use a language other than English (in my case, Korean), the length of the sentences in the content of the results is too short.

For example, in example1, when I ask "who is Messi?", the content length is quite long. However, in example2, when I ask the same question in Korean, the content part is shortened.

My question is, is there a way to output the entire content without shortening it? I need the full content because I use the search results as the retrieve values for RAG.

Additionally, are the sentences in the content part the entire text from the URL, or is there an algorithm that retrieves only part of the text?

Looking forward to your response! Thanks!

# example1
from tavily import TavilyClient
import os

TAVILY_API_KEY = os.getenv("TAVILY_API_KEY")
client = TavilyClient(api_key=TAVILY_API_KEY)

response = client.search("Who is Messi?")

response

>> {'query': 'Who is Messi?',
 'follow_up_questions': None,
 'answer': None,
 'images': [],
 'results': [{'title': 'Career of Lionel Messi - Wikipedia',
   'url': 'https://en.wikipedia.org/wiki/Career_of_Lionel_Messi',
   'content': "Lionel Messi's club career began with Barcelona, where he rose through the youth ranks, making his first-team debut in 2004.Over the next years, Messi became the club's all-time leading scorer, amassing numerous domestic and international accolades. During his tenure, Barcelona secured ten La Liga titles, seven Copa del Rey victories, and four UEFA Champions League victories.",
   'score': 0.79230356,
   'raw_content': None},
  {'title': "Lionel Messi | Biography, Barcelona, PSG, Ballon d'Or, Inter Miami ...",
   'url': 'https://www.britannica.com/biography/Lionel-Messi',
   'content': 'In early 2009 Messi capped off a spectacular 2008–09 season by helping FC Barcelona capture the club’s first “treble” (winning three major European club titles in one season): the team won the La Liga championship, the Copa del Rey (Spain’s major domestic cup), and the Champions League title. Messi’s play continued to rapidly improve over the years, and by 2008 he was one of the most dominant players in the world, finishing second to Manchester United’s Cristiano Ronaldo in the voting for the 2008 Ballon d’Or. At the 2014 World Cup, Messi put on a dazzling display, scoring four goals and almost single-handedly propelling an offense-deficient Argentina team through the group stage and into the knockout rounds, where Argentina then advanced to the World Cup final for the first time in 24 years. After Argentina was defeated in the Copa final—the team’s third consecutive finals loss in a major tournament—Messi said that he was quitting the national team, but his short-lived “retirement” lasted less than two months before he announced his return to the Argentine team. Messi helped Barcelona capture another treble during the 2014–15 season, leading the team with 43 goals scored over the course of the campaign, which resulted in his fifth world player of the year honour.',
   'score': 0.7444706,
   'raw_content': None},
  {'title': 'Lionel Messi - Wikipedia',
   'url': 'https://en.wikipedia.org/wiki/Lionel_Messi',
   'content': 'He scored twice in the last group match, a 3–2 victory over Nigeria, his second goal coming from a free kick, as they finished first in their group.[423] Messi assisted a late goal in extra time to ensure a 1–0 win against Switzerland in the round of 16, and played in the 1–0 quarter-final win against Belgium as Argentina progressed to the semi-final of the World Cup for the first time since 1990.[424][425] Following a 0–0 draw in extra time, they eliminated the Netherlands 4–2 in a penalty shootout to reach the final, with Messi scoring his team\'s first penalty.[426]\nBilled as Messi versus Germany, the world\'s best player against the best team, the final was a repeat of the 1990 final featuring Diego Maradona.[427] Within the first half-hour, Messi had started the play that led to a goal, but it was ruled offside. "[582] Moreover, several pundits and footballing figures, including Maradona, questioned Messi\'s leadership with Argentina at times, despite his playing ability.[583][584][585] Vickery states the perception of Messi among Argentines changed in 2019, with Messi making a conscious effort to become "more one of the group, more Argentine", with Vickery adding that following the World Cup victory in 2022 Messi would now be held in the same esteem by his compatriots as Maradona.[581]\nComparisons with Cristiano Ronaldo\nAmong his contemporary peers, Messi is most often compared and contrasted with Portuguese forward Cristiano Ronaldo, as part of an ongoing rivalry that has been compared to past sports rivalries like the Muhammad Ali–Joe Frazier rivalry in boxing, the Roger Federer–Rafael Nadal rivalry in tennis, and the Prost–Senna rivalry from Formula One motor racing.[586][587]\nAlthough Messi has at times denied any rivalry,[588][589] they are widely believed to push one another in their aim to be the best player in the world.[160] Since 2008, Messi has won eight Ballons d\'Or to Ronaldo\'s five,[590] seven FIFA World\'s Best Player awards to Ronaldo\'s five, and six European Golden Shoes to Ronaldo\'s four.[591] Pundits and fans regularly argue the individual merits of both players.[160][592] On 11 July, Messi provided his 20th assist of the league season for Arturo Vidal in a 1–0 away win over Real Valladolid, equalling Xavi\'s record of 20 assists in a single La Liga season from 2008 to 2009;[281][282] with 22 goals, he also became only the second player ever, after Thierry Henry in the 2002–03 FA Premier League season with Arsenal (24 goals and 20 assists), to record at least 20 goals and 20 assists in a single league season in one of Europe\'s top-five leagues.[282][283] Following his brace in a 5–0 away win against Alavés in the final match of the season on 20 May, Messi finished the season as both the top scorer and top assist provider in La Liga, with 25 goals and 21 assists respectively, which saw him win his record seventh Pichichi trophy, overtaking Zarra; however, Barcelona missed out on the league title to Real Madrid.[284] On 7 March, two weeks after scoring four goals in a league fixture against Valencia, he scored five times in a Champions League last 16-round match against Bayer Leverkusen, an unprecedented achievement in the history of the competition.[126][127] In addition to being the joint top assist provider with five assists, this feat made him top scorer with 14 goals, tying José Altafini\'s record from the 1962–63 season, as well as becoming only the second player after Gerd Müller to be top scorer in four campaigns.[128][129] Two weeks later, on 20 March, Messi became the top goalscorer in Barcelona\'s history at 24 years old, overtaking the 57-year record of César Rodríguez\'s 232 goals with a hat-trick against Granada.[130]\nDespite Messi\'s individual form, Barcelona\'s four-year cycle of success under Guardiola – one of the greatest eras in the club\'s history – drew to an end.[131] He still managed to break two longstanding records in a span of seven days: a hat-trick on 16 March against Osasuna saw him overtake Paulino Alcántara\'s 369 goals to become Barcelona\'s top goalscorer in all competitions including friendlies, while another hat-trick against Real Madrid on 23 March made him the all-time top scorer in El Clásico, ahead of the 18 goals scored by former Real Madrid player Alfredo Di Stéfano.[160][162] Messi finished the campaign with his worst output in five seasons, though he still managed to score 41 goals in all competitions.[161][163] For the first time in five years, Barcelona ended the season without a major trophy; they were defeated in the Copa del Rey final by Real Madrid and lost the league in the last game to Atlético Madrid, causing Messi to be booed by sections of fans at the Camp Nou.[164]',
   'score': 0.7350715,
   'raw_content': None},
  {'title': 'The life and times of Lionel Messi',
   'url': 'https://www.nytimes.com/athletic/4783674/2023/08/18/lionel-messi-profile-soccer/',
   'content': 'For Messi, it is major trophy number 44.. Despite turning 36 in June, he is as influential as ever. Here is the complete story of Lionel Andres Messi, widely regarded as one of the greatest ...',
   'score': 0.6631238,
   'raw_content': None},
  {'title': 'Lionel Messi | Paris Saint-Germain',
   'url': 'https://en.psg.fr/teams/first-team/squad/lionel-messi',
   'content': "Lionel\nMessi\nFirst Team\n30\nLionel\nMessi 30\nPlayer profile\nIt’s hard to find the words to describe a man that the football world knows simply as ‘The GOAT’. Lionel Messi: A Paris Saint-Germain legend\nMessi and Ramos play final match at the Parc\nLeo Messi finishes as Ligue 1’s top assist provider\nPictures of Lionel Messi with Paris Saint-Germain\nThanks, Leo!\n The all-time leading goal scorer for his country, the top scorer in the history of Spanish football, the top assist provider in the 21st century, Messi has broken all the records, but above all, inspired an entire generation of footballers around the world with his genius.\n Previous clubs\n2004-2021\nFC Barcelona\n778\nLionel Messi Related News\nLionel Messi's last game with Paris!\n Top goals: Paris-Clermont\nParis Saint-Germain\nServices\nLanguagues\nHelp\nJoin Us",
   'score': 0.62520933,
   'raw_content': None}],
 'response_time': 1.5}

# example2
from tavily import TavilyClient
import os

TAVILY_API_KEY = os.getenv("TAVILY_API_KEY")
client = TavilyClient(api_key=TAVILY_API_KEY)

response = client.search("메시는 누구야?")

response

>> {'query': '메시는 누구야?',
 'follow_up_questions': None,
 'answer': None,
 'images': [],
 'results': [{'title': "''쟤 누구야'' 10대 메시 처음 보고 전설이 내뱉은 한마디",
   'url': 'https://www.sportalkorea.com/news/view.php?gisa_uniq=2021032807561855&gomb=1',
   'content': "''쟤 누구야'' 10대 메시 처음 보고 전설이 내뱉은 한마디 ... 는 확실히 떡잎부터 남달랐던 괴물이었다. 메시는 축구사 최고로 평가받는다. 지난 2004년 바르셀로나를 통해 프로에 데뷔한 메시는 현재까지 768경기서 663골로 놀라운 득점 기록을 쓰고 있다. 34세 ...",
   'score': 0.9980808,
   'raw_content': None},
  {'title': '"쟤 누구야" 10대 메시 처음 보고 전설이 내뱉은 한마디 : 네이트 스포츠',
   'url': 'https://sports.news.nate.com/view/20210328n07730',
   'content': '지오 판 브롱크호스트에게 \'쟤 누구야\'라고 물어볼 만큼 아주 잘했었다"라고 돌아봤다. 지금은 바르셀로나 코치로 메시를 지도하는 라르손은 "메시는 현재 세계 최고의 선수가 됐다. 그가 한 모든 일은 엄청나다. 수년 동안 계속해서 같은 수준을 보여주는 건 ...',
   'score': 0.9901782,
   'raw_content': None},
  {'title': '"이제 넌 가만히 있고 알바가 올라갈 거야! 영어 괜찮지?"...메시가 인터 마이애미 동료에게 영어로 한 말',
   'url': 'https://yourfield.nexon.com/today/view/SK007_20240303_7101?type=all',
   'content': '메시는 바르셀로나 역대 최다 득점자다. 2021년 여름 바르셀로나와 재계약이 불발됐던 메시는 이후 파리 생제르맹에서 두 시즌을 보냈다. 지난해 7월 psg와 계약이 만료됐던 메시는 인터 마이애미에 입단했다. 메시는 미국 진출 후에도 마법을 일으켰다.',
   'score': 0.9854964,
   'raw_content': None},
  {'title': '메시, 21년간 뛴 바르셀로나 떠난다 - 조선일보',
   'url': 'https://www.chosun.com/sports/world-football/2021/08/07/PVH7XHZISVA7LFPI42UA74WYYM/',
   'content': '메시는 2000년 바르셀로나 유스팀과 계약했고, 2004년 10월 1군 무대에 데뷔했다. 바르셀로나에서 17시즌 778경기를 뛰면서 672골을 넣으며 구단 역대 최다 출장, 득점 기록을 세웠다. 정규리그 10회, uefa(유럽축구연맹) 챔피언스리그 4회 등 우승컵을 35번 들었다.',
   'score': 0.9783433,
   'raw_content': None},
  {'title': '메시: 결국 바르셀로나에 남는다 - BBC News 코리아',
   'url': 'https://www.bbc.com/korean/news-54038898',
   'content': 'BBC 스포츠 분석가, 앤디 웨스트. 메시는 결국 남는다. 하지만 이야기의 끝은 아직 멀었다. 메시는 인터뷰에서 바르셀로나 주제프 마리아 바르토메우 ...',
   'score': 0.9749169,
   'raw_content': None}],
 'response_time': 1.43}