Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add required information to DBP NA<->FE API for removal timeline support #3268

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

THISISDINOSAUR
Copy link
Contributor

@THISISDINOSAUR THISISDINOSAUR commented Sep 12, 2024

Task/Issue URL: https://app.asana.com/0/1203581873609357/1208164179397484/f
Tech Design URL:
Initial API additions: https://app.asana.com/0/481882893211075/1208203761791777/f
Parent - child api info and matching logic: https://app.asana.com/0/72649045549333/1208303545057273/f
Where to put the matching logic: https://app.asana.com/0/481882893211075/1208325118544624/f
CC:

Description:

  • Adds four new fields as per the first tech design. Also moves some of the construction of these objects to constructors.

  • Also removes an old date field that isn't officially in the api, and apparently varied what it was used for. It seemed unused on the native side, and the FE side doesn't know it exist.

  • It then also adds "parent child" matching logic so we can identify orphaned child records that will never be removed.

Steps to test this PR:

  1. In DataBrokerProtectionWebUIURLSettings, change the production URL to "https://use-devtesting27.duckduckgo.com/dbp", Alex made an FE build we can use to test
  2. In the DBP UI, click on individual records, this will show you things like the opt out submitted date
  3. Make sure these values make sense, e.g. we don't ever show 1970, the timeline follows linear time, nothing violates causality etc.

Definition of Done:


Internal references:

Pull Request Review Checklist
Software Engineering Expectations
Technical Design Template
Pull Request Documentation

addresses: extractedProfile.addresses?.map {DBPUIUserProfileAddress(addressCityState: $0) } ?? [],
alternativeNames: extractedProfile.alternativeNames ?? [String](),
relatives: extractedProfile.relatives ?? [String](),
foundDate: optOutJobData.createdDate.timeIntervalSince1970,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love that this information is gained from the optOutJobData, but what you can't see from looking just at this struct is if you see further above is that all these extracted profiles are taken from the opt out jobs the first place.
So I don't think there's anything I can do about it outside of making more serious challenges to our data architecture (which I'd love to do. In making this PR I really noticed how my issues with the DB scheme trickle all the way up to cause a lot of the things I don't like in the UI mapping stuff)

Copy link

github-actions bot commented Sep 18, 2024

Warnings
⚠️ PR has more than 500 lines of code changing. Consider splitting into smaller PRs if possible.

Generated by 🚫 dangerJS against c2d4d9f

}

extension DBPUIDataBrokerProfileMatch {
init(optOutJobData: OptOutJobData,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really like how long this construct got, but didn't have any nice ideas to simplify it, so taking suggestions!

@@ -22,36 +22,6 @@ import os.log

struct MapperToUI {

func mapToUI(_ dataBroker: DataBroker, extractedProfile: ExtractedProfile) -> DBPUIDataBrokerProfileMatch {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this has now moved into constructors on the individual UI objects

@@ -74,21 +44,39 @@ struct MapperToUI {
totalScans: totalScans,
scannedBrokers: partiallyScannedBrokers)

let matches = mapMatchesToUI(brokerProfileQueryData)
let matches = mapMatchesToUI(withoutDeprecated)

return .init(resultsFound: matches, scanProgress: scanProgress)
}

private func mapMatchesToUI(_ brokerProfileQueryData: [BrokerProfileQueryData]) -> [DBPUIDataBrokerProfileMatch] {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like how much this and the maintenance scans function duplicate each other. However I'm also afraid to touch them since there seem to be some subtle differences which I assume are deliberate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant