Skip to content
This repository has been archived by the owner on Sep 10, 2021. It is now read-only.

Added __slots__ for all classes (possible to reduce the memory) #37

Merged
merged 1 commit into from
Jul 28, 2020
Merged

Added __slots__ for all classes (possible to reduce the memory) #37

merged 1 commit into from
Jul 28, 2020

Conversation

divaltor
Copy link
Contributor

@divaltor divaltor commented Jul 27, 2020

#36

I tried adding slots to classes to reduce memory consumption. Since when parsing real yml files it may contain several thousand offers and creating classes without slots may have a bad effect on memory

chrome_0DeiDZ36LA

  • All tests are passed

  • No conflicts

  • Tests on real file

The first image is a variant with slots, the second without them. I wouldn't say the difference is too big, but it's still a little better than nothing. The test may not be too plausible due to the fact that I additionally process it after it has loaded into memory and perhaps because of this, the difference is almost imperceptible. The test was performed on a 110 MB file

vlc_Y0E2cPq7Ap
vlc_vZpamgUyDW

@codecov-commenter
Copy link

codecov-commenter commented Jul 27, 2020

Codecov Report

Merging #37 into master will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff            @@
##            master       #37   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           22        22           
  Lines         1147      1172   +25     
=========================================
+ Hits          1147      1172   +25     
Impacted Files Coverage Δ
yandex_market_language/models/age.py 100.00% <100.00%> (ø)
yandex_market_language/models/category.py 100.00% <100.00%> (ø)
yandex_market_language/models/condition.py 100.00% <100.00%> (ø)
yandex_market_language/models/currency.py 100.00% <100.00%> (ø)
yandex_market_language/models/dimensions.py 100.00% <100.00%> (ø)
yandex_market_language/models/feed.py 100.00% <100.00%> (ø)
yandex_market_language/models/gift.py 100.00% <100.00%> (ø)
yandex_market_language/models/offers.py 100.00% <100.00%> (ø)
yandex_market_language/models/option.py 100.00% <100.00%> (ø)
yandex_market_language/models/parameter.py 100.00% <100.00%> (ø)
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b7e5bc7...e3b48c6. Read the comment docs.

@stefanitsky
Copy link
Owner

@divaltor good job, but what about what we talked about earlier to use an iterator? Did you tried that? I think i can merge that, but i think this is not closes #36

@divaltor
Copy link
Contributor Author

@stefanitsky I'm thinking how to do this so as not to break backward compatibility

@stefanitsky
Copy link
Owner

@stefanitsky I'm thinking how to do this so as not to break backward compatibility

Do you need to change models for this? If not, you can make another method and name it like iter_parse.

@stefanitsky stefanitsky merged commit 1b05521 into stefanitsky:master Jul 28, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants