forked from openedx/edx-documentation
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: Script to create .po files for Transifex Translations (openedx#…
…2041) * feat: Add a script for creating i18n .po files i18n .po files are necessary to create Transifex Projects. This script was originally written by @ghassanmas. This script first turns the docs into .pot files and then via sphinx-intl turns them into .po files. * feat: Add python script to edit .tx/config Transifex only allows uniquely named files. Since we often use similarly named files, such as `index.rst`, this would replace each file in transifex with the similarly named one, which is not ideal. Instead, a script changes the titles of the sections in the config file to make them unique. * feat: Add i18n support to Makefile * feat: Update the requirements to support i18n Python reqs such as sphinx-intl and transifex-client are necessary to make the translation files and upload them to transifex. * fix: Remove CI test * fix: Add default Project Name, update exit message * fix: Catch certain common errors `set -eoux pipefail` was added to catch certain common errors. And the todo item was removed. * docs: Add docstring with explanation and example * fix: Remove x option The x option prints all lines run in script, making the output ridiculously messy. * feat: Sys argv replaced with Click arguments * fix: Make get more pythonic with fallback In addition, the script was reformatted as it all falls under the function now. * fix: Add maxsplit to limit # of splits The maxsplit has been set to 1, meaning it splits into 2 parts, only splitting on the first '.'. * fix: Add delimiter between dir and file name The delimiter used to split up the project path is also used to separate the file name from the delimited project path. Co-authored-by: Ghassan Maslamani <[email protected]>
- Loading branch information
1 parent
0410ace
commit 96e99a2
Showing
7 changed files
with
207 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -35,3 +35,6 @@ autodeploy.properties | |
|
|
||
| ### Doc health data | ||
| health.csv | ||
|
|
||
| ### Translations and Transifex | ||
| .tx/ | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| #!/bin/bash | ||
|
|
||
| # This script purpose is to ease process needed to support i18n | ||
| # for the doc. It does that by first generating pot from the | ||
| # rst files of a selected projects using sphinx to generate gettext | ||
| # (which outputs the pot files) and then the next step | ||
| # is to use sphinx-intl to write to the ./tx/config file, hence | ||
| # this script cerates the ./tx/config file. Note: if the file, | ||
| # already exists it will be overwritten!. | ||
| # This script assumes sphinx-intl is already installed as well, | ||
| # It can be installed by `pip install sphinx-intl` | ||
| # For more information, about sphinx-intl please visit: | ||
| # https://www.sphinx-doc.org/en/master/usage/advanced/intl.html | ||
|
|
||
|
|
||
| set -eou pipefail | ||
|
|
||
|
|
||
| IFS= read -r -p "Enter the project name as it appears in Transifex[open-edx-documentation-project]: " PROJECT_NAME | ||
| PROJECT_NAME=${PROJECT_NAME:-open-edx-documentation-project} | ||
|
|
||
| # Root directory | ||
| BASE_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) | ||
| # Config file | ||
| TX_CONFIG_FILE=$BASE_DIR/.tx/config | ||
|
|
||
| # Success exit message, wrapped in green color. | ||
| EXIT_MESSAGE="\033[0;32mYou should now be able to push the files using tx client \n | ||
| Typically this done by 'tx push -s', please note: you should have \n | ||
| already installed tx client and configured it such that there is \n | ||
| a project called '$PROJECT_NAME' in the organization \n | ||
| and that you already have authenticated with in '~/.transifexrc', \n | ||
| for more information, please visit: \n | ||
| https://docs.transifex.com/client/client-configuration.\033[0m" | ||
|
|
||
| # Preparing the config file: | ||
| rm -rf .tx | ||
| mkdir $BASE_DIR/.tx | ||
| touch $TX_CONFIG_FILE | ||
| echo "[main]" >> $TX_CONFIG_FILE | ||
| echo "host = https://www.transifex.com" >> $TX_CONFIG_FILE | ||
|
|
||
|
|
||
| # The set of books to run translations for. | ||
| # Currently this is only a subset of the books in this repo because we are in the middle of | ||
| # assesing which books will remain a part of openedx and which ones are actually specific | ||
| # to edx.org | ||
| projects=( | ||
| "en_us/open_edx_students" | ||
| "en_us/open_edx_course_authors" | ||
| "en_us/open_edx_release_notes" | ||
| ) | ||
|
|
||
| # Iterating over each project defined above, to generate the pot the files | ||
| # and then mapping the files accordingly in the tx config file | ||
| for project in "${projects[@]}"; do | ||
| cd $BASE_DIR/$project | ||
| echo "--> Start generating the pot files for ${project#'en_us/'}" | ||
|
|
||
| # Extract translatable messages into pot files | ||
| make gettext | ||
|
|
||
| # Adding the pot files to tx config file | ||
| echo "--> Start writing tx configuration for the ${project#'en_us/'}" | ||
|
|
||
| cd $BASE_DIR | ||
| sphinx-intl update-txconfig-resources \ | ||
| --pot-dir $BASE_DIR/$project/build/locale/ \ | ||
| --locale-dir $BASE_DIR/$project/locales/ \ | ||
| --transifex-project-name "$PROJECT_NAME" | ||
|
|
||
| python tx_config_fix.py "$project" | ||
|
|
||
| done | ||
|
|
||
| echo -e ${EXIT_MESSAGE} | ||
|
|
||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| from configparser import ConfigParser | ||
|
|
||
| import click | ||
|
|
||
|
|
||
| @click.command() | ||
| @click.argument("project_path") | ||
| def delimit_sections(project_path): | ||
| """ | ||
| This script changes the section titles of the tx config file to be more unique based | ||
| on the directory of the file that is used to make the original section title. | ||
| The script loops through the sections of the tx config and changes the sections that | ||
| match the project_path argument variable. The script then adds a delimited version of | ||
| the directory to the section title making the title more unique. These section titles | ||
| become the file names on Transifex, and if they are not unique, they overwrite | ||
| eachother. | ||
| Example: | ||
| If we had a file "example/index.rst" that we wanted to translate, in tx config we | ||
| would have a section with the title [index] and the source_file: "example/index.rst". | ||
| But this filename is common in Sphinx Projects. This script would rename the section | ||
| title to be [example--title], keeping all other values under it untouched. | ||
| """ | ||
| delimiter = "--" | ||
| project_path_friendly = project_path.replace("/", delimiter) | ||
|
|
||
| config_path = "./.tx/config" | ||
|
|
||
| config = ConfigParser() | ||
| config.read(config_path) | ||
|
|
||
| for section_title in config.sections(): | ||
| source_file_path = config.get(section_title, "source_file", fallback=None) | ||
| if source_file_path and source_file_path.startswith(project_path): | ||
| split_section_title = section_title.split(".", maxsplit=1) | ||
| split_section_title[1] = ( | ||
| project_path_friendly + delimiter + split_section_title[1] | ||
| ) | ||
| new_section_title = ".".join(split_section_title) | ||
| config.add_section(new_section_title) | ||
| for option_name in config.options(section_title): | ||
| config.set( | ||
| new_section_title, option_name, config[section_title][option_name] | ||
| ) | ||
| config.remove_section(section_title) | ||
|
|
||
| with open(config_path, "w") as file: | ||
| config.write(file) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| delimit_sections() |