X Tutup

COLLECTED BY

Organization: Archive Team

Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.

History is littered with hundreds of conflicts over the future of a community, group, location or business that were "resolved" when one of the parties stepped ahead and destroyed what was there. With the original point of contention destroyed, the debates would fall to the wayside. Archive Team believes that by duplicated condemned data, the conversation and debate can continue, as well as the richness and insight gained by keeping the materials. Our projects have ranged in size from a single volunteer downloading the data to a small-but-critical site, to over 100 volunteers stepping forward to acquire terabytes of user-created data to save for future generations.

The main site for Archive Team is at archiveteam.org and contains up to the date information on various projects, manifestos, plans and walkthroughs.

This collection contains the output of many Archive Team projects, both ongoing and completed. Thanks to the generous providing of disk space by the Internet Archive, multi-terabyte datasets can be made available, as well as in use by the Wayback Machine, providing a path back to lost websites and work.

Our collection has grown to the point of having sub-collections for the type of data we acquire. If you are seeking to browse the contents of these collections, the Wayback Machine is the best first stop. Otherwise, you are free to dig into the stacks to see what you may find.

The Archive Team Panic Downloads are full pulldowns of currently extant websites, meant to serve as emergency backups for needed sites that are in danger of closing, or which will be missed dearly if suddenly lost due to hard drive crashes or server failures.

Collection: Archive Team: The Github Hitrub

This collection is a set of Github repository archives from two major sets: A panic grab upon the acquisition by Microsoft, and a larger, ongoing set of Pretty Much Everything.

TIMESTAMPS

The Wayback Machine - https://web.archive.org/web/20200906045149/https://github.com/brianc/node-postgres/pull/1788/

brianc / node-postgres

Sponsor

Sponsor brianc/node-postgres
Watch 187
Star 8.8k
Fork 945

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include raw SQL statement and values in query errors #1788

Open

marshall007 wants to merge 3 commits into brianc:master from marshall007:marshall_881

marshall007 commented Dec 12, 2018

Fixes #881


        Include raw SQL statement and values in query errors (#881)

624a60d

joelmukuthu reviewed

lib/utils.js Outdated

  
        function handleQueryError (err, query) {

          err.sql = {

            text: query.text,

            values: query.values

joelmukuthu Dec 21, 2018

Including values introduces a security issue. This could lead to user data e.g. passwords being logged in production. Perhaps this should be configurable or removed altogether.

marshall007 Dec 21, 2018 •

edited

Author

@joelmukuthu agreed, do you think we should just make the population of err.sql.values configurable, err.sql entirely, or have fine-grained options for both?


        Configurable inclusion of SQL statement in errors

dd7352c

Author

marshall007 commented Dec 21, 2018

I've addressed @joelmukuthu's concern in dd7352c. There is a new config option, sql_in_error (totally open to a better name there), which accepts the following values:

false: none (default, current behavior)
true: text and values from query
"text": only text from query


        Explicitly test exclusion of `values` from query errors

b97b4d6

mledom commented Feb 11, 2019

This is great, I was actually trying to find a way to get that information.

Owner

brianc commented Feb 20, 2019

I'm not a fan of doing this - there's a security issue here in that we could be causing apps to accidentally log out sensitive information (e.g. a hashed password in the query params). It also is something that can/should be handled by a higher level library or your own application code if this kind of logging is required. Our responsibility as a database driver is to propagate the error back to the caller w/ the details included from the PostgreSQL backend.

Contributor

abenhamdine commented Feb 27, 2019 •

edited

there's a security issue here in that we could be causing apps to accidentally log out sensitive information

It would be indeed unacceptable in many businesses (including our) and uncompliant with European regulation on personal data protection (GDPR).
However, it would be useful for debugging purpose, could you allow to enable it only with an option ?

Author

marshall007 commented Mar 4, 2019

... there's a security issue here in that we could be causing apps to accidentally log out sensitive information (e.g. a hashed password in the query params).

@brianc because the new behavior is strictly opt-in, I tend to disagree here. I'd be happy to add some logic that explicitly warns the user of your concern when NODE_ENV === 'production' and sql_in_error !== false. That would seem like a reasonable middle ground?

Owner

brianc commented Mar 6, 2019

I think the best approach here is to log your query text & sql out in your own application if you need to see it in your logs. I'm not comfortable putting code in that has the possible to leak secure information...even less so if it's non-compliant with regulation. I don't see a reason this has to be the responsibility of node-postgres.

charmander mentioned this pull request

feat: extend errors for trackability #1925

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

You can’t perform that action at this time.

X Tutup