Jump to content


WoT Dossier Tool


  • Please log in to reply
343 replies to this topic

AlongTheMilkyWays #21 Posted 28 August 2011 - 12:47 PM

    Junior Sergeant

  • Player
  • 6058 battles
  • 131
  • Member since:
    03-29-2011

View PostSnib, on 28 August 2011 - 12:14 PM, said:

Site looks nice, but was completely non-functional to me in both Opera 12 and Firefox 6, didn't try another browser.
The website requires javascript to work, you might have it disabled or using a NoScript plugin.

View PostSnib, on 28 August 2011 - 12:14 PM, said:

Question about the profiles: I saw you were storing them, but at least on the exposed web frontend for that archive it appears they are stored with a consecutive numbering system.
The IDs are not consecutive, they are selected randomly by the datastore. It just happens to cluster them around certain intervals.

View PostSnib, on 28 August 2011 - 12:14 PM, said:

Was hoping/suggesting, that internally you were storing them with a unique identifier and update them when they get uploaded again
All of the submitted data is anonymous and not published.

Dossier data is also immutable, so if you upload an updated document, it will get a new id (uploaded documents are also hashed, so if you upload the same data, you will get the same id).

Currently I cannot connect a dossier with its play's name. I could add authentication using Google Accounts and assign dossiers to that, that would allow to version the dossier file.

View PostSnib, on 28 August 2011 - 12:14 PM, said:

the second suggestion to add an overall stats view that combines the data from all the profiles ever uploaded.
Well, aggregation would add some value by showing average performance of certain tanks, while not linking to the source dossiers, I will look into it in the future.

Edit: The major problem with aggregation is that anyone can easily edit the file and put fake values, thus the dossier data cannot be trusted and aggregation would produce garbage results.

Snib #22 Posted 28 August 2011 - 02:00 PM

    First Sergeant

  • Beta Tester
  • 18888 battles
  • 4,911
  • Member since:
    07-16-2010

View PostAlongTheMilkyWays, on 28 August 2011 - 12:47 PM, said:

The website requires javascript to work, you might have it disabled or using a NoScript plugin.
Nope, enabled in Opera and no plugins installed for FF.

View PostAlongTheMilkyWays, on 28 August 2011 - 12:47 PM, said:

Dossier data is also immutable, so if you upload an updated document, it will get a new id (uploaded documents are also hashed, so if you upload the same data, you will get the same id).
You could ID them via the upload filename, unless the user modifies that one on purpose before uploading it should be unique (or as unique as hashes go).

View PostAlongTheMilkyWays, on 28 August 2011 - 12:47 PM, said:

Edit: The major problem with aggregation is that anyone can easily edit the file and put fake values, thus the dossier data cannot be trusted and aggregation would produce garbage results.
Yes, of course, as always in programming, never trust the input, but while figuring out the format of that file isn't too hard, you will need some background in coding to do it, so not every whiner on the forum is going to, which might make it worth your time to add an aggregate view. Anyway, was just a suggestion, good luck with a site!

AlongTheMilkyWays #23 Posted 28 August 2011 - 04:46 PM

    Junior Sergeant

  • Player
  • 6058 battles
  • 131
  • Member since:
    03-29-2011

View PostSnib, on 28 August 2011 - 02:00 PM, said:

Nope, enabled in Opera and no plugins installed for FF.
Installed Opera 12 Pre-Alpha and it works there as well. Sorry, I can't reproduce it.

View PostSnib, on 28 August 2011 - 02:00 PM, said:

You could ID them via the upload filename, unless the user modifies that one on purpose before uploading it should be unique (or as unique as hashes go).
Storage isn't an issue, I see no value with changing the data with something else, particularly that the price of it would be loss of anonymity.

View PostSnib, on 28 August 2011 - 02:00 PM, said:

Yes, of course, as always in programming, never trust the input, but while figuring out the format of that file isn't too hard, you will need some background in coding to do it, so not every whiner on the forum is going to, which might make it worth your time to add an aggregate view. Anyway, was just a suggestion, good luck with a site!
A tutorial about the format is available at http://wot.ctocopok.ru/ , so anybody can make a tool for that.

Also there was a thread, where a moderator classified this app as "fishing site" - http://forum.worldof...2-estadisticas/ . To prove that is a false positive, I can make a script that will take dossier data in JSON (check F.A.Q. page how to retrieve that) (they do not contain any text or any user-entered content) and generate a file, that exactly byte-to-byte matches the original dossier file.

But looking at the options how to implement aggregation, using mean values instead of averages would be a good defence against this.

Snib #24 Posted 28 August 2011 - 05:15 PM

    First Sergeant

  • Beta Tester
  • 18888 battles
  • 4,911
  • Member since:
    07-16-2010
Thanks for checking the browser issue, didn't throw any JS errors on my end either, but that's as far as I looked into it. I might have blocked some google scripts in the past because of the ever present GA and maybe that's interfering with your code.

Not too fussed tbh because I have my own local decoder for the cache files. That also means that I know that there's basically nothing but numbers inside of the file, and the only thing linking it to your account is the hashed filename.

Complaining about anonymity is quite besides the point anyway since most of the data inside of the file is available to the public already via the ingame profile view. I just thought that having aggregate data would be interesting in view of the only other aggregate data source we have is the website profiles which are limited to number of battles and wins.

Snib #25 Posted 28 August 2011 - 08:43 PM

    First Sergeant

  • Beta Tester
  • 18888 battles
  • 4,911
  • Member since:
    07-16-2010
Quick update: It was the GA block I had in place, without that site loads fine.

AlongTheMilkyWays #26 Posted 28 August 2011 - 09:31 PM

    Junior Sergeant

  • Player
  • 6058 battles
  • 131
  • Member since:
    03-29-2011

View PostSnib, on 28 August 2011 - 08:43 PM, said:

Quick update: It was the GA block I had in place, without that site loads fine.
It appears that the script loader I'm using does load all of the scripts, but does not execute the initialization function at the end, that's intended to be ran once they finish loading. A simple skipping of uninitialized ga.js is not enough, because the script loader is waiting for ga.js to execute, that's not gonna happen for the few people. Moved loading of ga.js to end of the list, now it will be working.

Balou_2004 #27 Posted 29 August 2011 - 08:13 AM

    Lance-corporal

  • Beta Tester
  • 7772 battles
  • 98
  • Member since:
    01-14-2011
Nice Work.
Thanks a lot and thumbs up !  :Smile_great:

Theurge #28 Posted 01 September 2011 - 04:03 AM

    Private

  • Player
  • 0 battles
  • 6
  • Member since:
    09-01-2011
This is a great tool! It'd be nice if there was a way to filter out the lower-tier tanks. For example, I don't really care about how many A-20s and M2 Mediums I've destroyed, I'm only interested in tier 7+. A filter by tier would be a great addition!

Thanks for your efforts!

AlongTheMilkyWays #29 Posted 01 September 2011 - 10:46 PM

    Junior Sergeant

  • Player
  • 6058 battles
  • 131
  • Member since:
    03-29-2011

View PostTheurge, on 01 September 2011 - 04:03 AM, said:

This is a great tool! It'd be nice if there was a way to filter out the lower-tier tanks. For example, I don't really care about how many A-20s and M2 Mediums I've destroyed, I'm only interested in tier 7+. A filter by tier would be a great addition!

Thanks for your efforts!
Added something like that, take a look in the frag-counts tab.

Thanks for feedback :)

war4peace #30 Posted 08 September 2011 - 02:11 AM

    First Sergeant

  • Translator
  • 14664 battles
  • 4,802
  • Member since:
    01-31-2011
Wow, great tool, really nice to see something like that.

I am offering my help in generating some nice reports with graphs and possibly expand the usability of this project. We all know people like graphical stuff :)
Now I am good at doing stuff in VBA and Excel is my very close friend, but honestly I can't find time to implement a JSON parser myself. Also, some tools that I use to generate nice reports (e.g. Tableau Software) prefer input to be something easily readable and widely available, such as XLS format.

What I found is a library that enables JSON-to-XLS conversion. it's located here. Would there be a chance for you to implement the "download as XLS" functionality? I can pick up raw data from there and develop a nice VBA script that would generate well-looking reports for everyone to see.

Please let me know if interested. I'll look for free online report generators in the meantime.

Theurge #31 Posted 08 September 2011 - 04:13 AM

    Private

  • Player
  • 0 battles
  • 6
  • Member since:
    09-01-2011

View PostAlongTheMilkyWays, on 01 September 2011 - 10:46 PM, said:

Added something like that, take a look in the frag-counts tab.

Thanks for feedback :)

Another suggestion for your awesome tool.

It'd be nice to have the ability to update your dossier, rather than having to create a new dossier each time you want to upload the latest data file. So far I've created six dossiers, and not only do I feel guilty about polluting the site with out of date data, but it's mildly annoying that I have to create a new bookmark to the new dossier page each time. Seems like it should be rather easy to incorporate the ability to upload a new file to the existing dossier. If such an ability were enabled, another benefit might be the ability to easily compare the before/after dossiers to gauge your progress, perhaps with highlighted cells (green cell would indicate improvement/addition, red cell would mean decrease).

And a question: why is the tool strictly web-based? A standalone executable would be even better. I only have a general understanding of programming, so my guess would be that you don't speak the right language to code Windows programs? But wait, I thought this was coded in Java--shouldn't you be able to port that with ease?

Anyway, let me thank you once more for the awesome program!

o0SMoOKE0o #32 Posted 08 September 2011 - 08:01 PM

    Lance-corporal

  • Player
  • 14263 battles
  • 93
  • Member since:
    04-01-2011
Hi,

big thumbs of the work you put into this tool !!

Just wondering if it would be possible to make the left (or right) column hoover in the 'Frag Counts' screen ?
This way if you side scroll with the bar on top to scroll through the different tanks you destroyed you can still see with what tank you destroyed them with. (At the moment you can only see it of the most left and most right destroyed tanks)

Maybe also possible to put a number in the square you use to change the 'Minimum fragged tank tier' ? So if you scroll it left or right the number in the square you drag will show the tier you are currently on ?

Please keep up the good work !

n0rdi #33 Posted 11 September 2011 - 01:26 AM

    Senior Sergeant

  • Player
  • 19066 battles
  • 581
  • Member since:
    04-09-2011
ignore it (my post)

AlongTheMilkyWays #34 Posted 11 September 2011 - 10:27 AM

    Junior Sergeant

  • Player
  • 6058 battles
  • 131
  • Member since:
    03-29-2011
I've added some requested features over the past few days. so I'll summarize now.

View Postwar4peace, on 08 September 2011 - 02:11 AM, said:

I am offering my help in generating some nice reports with graphs and possibly expand the usability of this project. We all know people like graphical stuff :)
Now I am good at doing stuff in VBA and Excel is my very close friend, but honestly I can't find time to implement a JSON parser myself. Also, some tools that I use to generate nice reports (e.g. Tableau Software) prefer input to be something easily readable and widely available, such as XLS format.

I've added a feature to generate CSV ( http://wot.ctocopok.ru/ is a better option for that through), it'll give easier access to apply spreadsheet software. Adding charts is obviously a clear to-do for the future.

Generating reports would be nice, having CSV available, anyone can to that locally. From the web-side, this could work by having a template document (in XML) that is filled with dossier data and linked for the user to download.

View Postwar4peace, on 08 September 2011 - 02:11 AM, said:

What I found is a library that enables JSON-to-XLS conversion. it's located here. Would there be a chance for you to implement the "download as XLS" functionality? I can pick up raw data from there and develop a nice VBA script that would generate well-looking reports for everyone to see.
JSON is a tree-based format, so not every document is suitable to be transformed into a flat table. Generating CSVs is easy; hopefully, it will give you something usefull to experiment with, if you wish.

View PostTheurge, on 08 September 2011 - 04:13 AM, said:

It'd be nice to have the ability to update your dossier, rather than having to create a new dossier each time you want to upload the latest data file. So far I've created six dossiers, and not only do I feel guilty about polluting the site with out of date data, but it's mildly annoying that I have to create a new bookmark to the new dossier page each time. Seems like it should be rather easy to incorporate the ability to upload a new file to the existing dossier.
Dossiers take little space and the oldest one can be deleted, but bookmarking is a good argument, so I've added it.

View PostTheurge, on 08 September 2011 - 04:13 AM, said:

If such an ability were enabled, another benefit might be the ability to easily compare the before/after dossiers to gauge your progress, perhaps with highlighted cells (green cell would indicate improvement/addition, red cell would mean decrease).
Yes, that would be a nice feature for the future.

View PostTheurge, on 08 September 2011 - 04:13 AM, said:

And a question: why is the tool strictly web-based? A standalone executable would be even better. I only have a general understanding of programming, so my guess would be that you don't speak the right language to code Windows programs? But wait, I thought this was coded in Java--shouldn't you be able to port that with ease?
Desktop apps are a thing of the past ;)http://prism.mozillalabs.com/ could give a more desktop look. Also it's written in Python :) But yes, a desktop app would be more personalized and have the advantage to constantly monitor the file without the need to manually reuploading it. Anyone who's willing can develop it, in fact they can use http://wot-dossier.a...dossier-to-json (from http://wot-dossier.a...pspot.com/#!faq ) for prototyping it.

View Posto0SMoOKE0o, on 08 September 2011 - 08:01 PM, said:

Just wondering if it would be possible to make the left (or right) column hoover in the 'Frag Counts' screen ?
This way if you side scroll with the bar on top to scroll through the different tanks you destroyed you can still see with what tank you destroyed them with. (At the moment you can only see it of the most left and most right destroyed tanks)

Maybe also possible to put a number in the square you use to change the 'Minimum fragged tank tier' ? So if you scroll it left or right the number in the square you drag will show the tier you are currently on ?
I've tried a few options, fixing of the column works by gutting the table's structure, so the sorting plugin does not apply easily. I do not think redeveloping and combining the two would an efficient use of time, so I've added better filtering options to select the columns you wish to see. Hopefully that should be less of a pain to use.

nafnist #35 Posted 11 September 2011 - 11:56 AM

    Senior Sergeant

  • Beta Tester
  • 10462 battles
  • 603
  • [NAF] NAF
  • Member since:
    11-20-2010
Awesome Job with the Tool!

.. but perhaps you should be more clear (meaning; simple), about what this Tool does.

Perhaps a Screenshot or a describing Headline.

Because this great Tool could be overlooked in the rush of internets gaming forum fansite debates

AlongTheMilkyWays #36 Posted 11 September 2011 - 12:12 PM

    Junior Sergeant

  • Player
  • 6058 battles
  • 131
  • Member since:
    03-29-2011

View Postnafnist, on 11 September 2011 - 11:56 AM, said:

.. but perhaps you should be more clear (meaning; simple), about what this Tool does.

Perhaps a Screenshot or a describing Headline.

Because this great Tool could be overlooked in the rush of internets gaming forum fansite debates

Yes, good suggestion, a screenshot indeed would be informative. However, if a user can't focus for a minute to read the description, it's unlikely that he/she will be interested in staring at a screen full of numbers :)

The app spreads virally, so new users are more likely to trust the user who's linking it, instead of app's title page. This is not a business, so there is no goal of making everyone, including their mom, to upload their dossiers :)

nafnist #37 Posted 11 September 2011 - 02:30 PM

    Senior Sergeant

  • Beta Tester
  • 10462 battles
  • 603
  • [NAF] NAF
  • Member since:
    11-20-2010
It is not obvious for many what this app does, is what I was trying to say.

This tool gives you alot more statistics than what is shown on players profile, like what tanks you need to destroy in-order to get Master Tanker medal ect. ect.

AlongTheMilkyWays #38 Posted 12 September 2011 - 07:53 PM

    Junior Sergeant

  • Player
  • 6058 battles
  • 131
  • Member since:
    03-29-2011

View Postnafnist, on 11 September 2011 - 02:30 PM, said:

It is not obvious for many what this app does, is what I was trying to say.

This tool gives you alot more statistics than what is shown on players profile, like what tanks you need to destroy in-order to get Master Tanker medal ect. ect.
Ok, added a more extensive description.

war4peace #39 Posted 16 September 2011 - 03:35 PM

    First Sergeant

  • Translator
  • 14664 battles
  • 4,802
  • Member since:
    01-31-2011
I've tested your CSV output with Tableau public; brilliant.
Now I have data in a nice format and good for making charts out of it.
Excellent job, mate! Kudos!

Phalynx #40 Posted 20 September 2011 - 10:24 AM

    First Sergeant

  • Player
  • 7557 battles
  • 1,122
  • [MTW] MTW
  • Member since:
    07-14-2011
Are you saving all updates to a single dossier?
As I'm using your tool to analyze my stats on a regular basis with the update function, I would like to see a chart on my progress. Any chance to see such a function implemented?




1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users