Jump to content


Replay analysis - what does this tell us about the MM?

Lets do science statistics

  • Please log in to reply
228 replies to this topic

Baldrickk #21 Posted 08 July 2018 - 09:59 AM

    Field Marshal

  • Player
  • 29322 battles
  • 13,387
  • [-TAH-] -TAH-
  • Member since:
    03-03-2013

View PostDorander, on 08 July 2018 - 09:57 AM, said:

3 pages of "MM is rigged" arguments, then another 20 of "you have no evidence" arguments.

 

And that's with me posting to make this tool availiable... sigh

CmdRatScabies #22 Posted 08 July 2018 - 10:01 AM

    Colonel

  • Player
  • 35441 battles
  • 3,662
  • [-MM] -MM
  • Member since:
    10-12-2015

View PostDorander, on 08 July 2018 - 09:57 AM, said:

 

 

It might seem like kicking in an open door and for any reasonable person it is, but given the prevalence of conspiracy threads by people who are convinced the game is rigged against them, the door isn't as open as people would like to think. Even solid and reliable assumptions benefit from testing and the resulting evidence.

 

Just check the recent "mm is rigged" thread that's been going on these past few weeks, it started with 3 pages of "MM is rigged" arguments, then another 20 of "you have no evidence" arguments.

 

Yes but clearly Muffin has been marked by WG as a "winner" so his results are rigged to win.  :)

CoDiGGo #23 Posted 08 July 2018 - 10:04 AM

    Warrant Officer

  • Player
  • 14754 battles
  • 566
  • [MOARR] MOARR
  • Member since:
    05-10-2015

View PostBaldrickk, on 08 July 2018 - 09:51 AM, said:

 

I made a few changes last night that fixed a couple of things, give me a second to upload a new release.

 

edit: Done.  https://github.com/B...ases/tag/v0.1.2 there is now a .exe that is up to date with the code.

 

ok, I have the exe, and now? 

Baldrickk #24 Posted 08 July 2018 - 10:13 AM

    Field Marshal

  • Player
  • 29322 battles
  • 13,387
  • [-TAH-] -TAH-
  • Member since:
    03-03-2013

View PostCoDiGGo, on 08 July 2018 - 10:04 AM, said:

 

ok, I have the exe, and now? 

 

updated the post above with step by step instructions, are they ok?

Bigtime_Alarm #25 Posted 08 July 2018 - 10:14 AM

    Staff Sergeant

  • Player
  • 17415 battles
  • 333
  • Member since:
    05-14-2013
But MM likes Lord Muffin, it hates me :teethhappy:

Balc0ra #26 Posted 08 July 2018 - 10:44 AM

    Field Marshal

  • Player
  • 63764 battles
  • 15,018
  • [WALL] WALL
  • Member since:
    07-10-2012

View PostCatn1p, on 08 July 2018 - 04:51 AM, said:

It tells that if you get a streak of terrible teams for 100 games and then a streak of great teams for 100 games then you have gotten average teams for 200 games.

 

Friggin’ genius.

 

Or a way of proving, rather than saying/whining. It is rather even vs rigged one way. But as you only play one tank by the looks of it... alt account or not. On a tier where you have to carry 9 out 10 games. I see why you get such big "fail" waves. 

 

View Postr0f, on 08 July 2018 - 09:22 AM, said:

What does this tell us about you? :trollface:

 

 

Or rather you? As I would think this would be right up your alley going by what you have complained about of late. 

 

 

 



Baldrickk #27 Posted 08 July 2018 - 11:05 AM

    Field Marshal

  • Player
  • 29322 battles
  • 13,387
  • [-TAH-] -TAH-
  • Member since:
    03-03-2013

View PostBaldrickk, on 08 July 2018 - 09:39 AM, said:

 

So, a moving average instead?

 

Regarding more one-sided results, I did that test with my replays a while back when I first did that graph, and there was virtually no difference.  It's buried in the MM thread somewhere... 
Finally found it, post 3731...

This is with about 6k battles either side iirc

Your 5.5k replays before and after MM change. 

I haven't done any normalisation of tiers in any of the pre/post graphs.  We know that it can affect the skill averages can vary based on tier, it's presumable that the result distribution could too.

Still, it shows that there isn't actually any significant deviation.


Edited by Baldrickk, 08 July 2018 - 11:12 AM.


Flubber42 #28 Posted 08 July 2018 - 11:15 AM

    Private

  • Player
  • 26448 battles
  • 34
  • [BC28] BC28
  • Member since:
    12-14-2014

Hold on, why arent you including LordMuffin in the rating of the friendly team?

 

  • "Position on the Y axis is the average rating of the friendly team (not including LordMuffin himself)"

 



LordMuffin #29 Posted 08 July 2018 - 11:24 AM

    Field Marshal

  • Player
  • 47472 battles
  • 10,573
  • [-GLO-] -GLO-
  • Member since:
    06-21-2011

View PostFlubber42, on 08 July 2018 - 11:15 AM, said:

Hold on, why arent you including LordMuffin in the rating of the friendly team?

 

  • "Position on the Y axis is the average rating of the friendly team (not including LordMuffin himself)"

 

Because we check a random players (me) team mates compared to enemy teams players with the intent to see if there is a skill difference between the teams, and if yes, how big.


Edited by LordMuffin, 08 July 2018 - 11:25 AM.


Long_Range_Sniper #30 Posted 08 July 2018 - 11:26 AM

    General

  • Player
  • 30321 battles
  • 8,176
  • [-MM] -MM
  • Member since:
    04-04-2011

Bakdrickk, I've followed the instructions step by step but get told that replay_analyser is not recognized as the name of a cmdlet. Also I backup my replays to another drive do I've pointed it at that folder. Any ideas, as I can't get it to work?

 

:medal:+10 though

 

Edit: Might have sussed it...stand by!


Edited by Long_Range_Sniper, 08 July 2018 - 11:30 AM.


Baldrickk #31 Posted 08 July 2018 - 11:31 AM

    Field Marshal

  • Player
  • 29322 battles
  • 13,387
  • [-TAH-] -TAH-
  • Member since:
    03-03-2013

View PostLong_Range_Sniper, on 08 July 2018 - 11:26 AM, said:

Bakdrickk, I've followed the instructions step by step but get told that replay_analyser is not recognized as the name of a cmdlet. Also I backup my replays to another drive do I've pointed it at that folder. Any ideas, as I can't get it to work?

 

:medal:+10 though

 

Edit: Might have sussed it...stand by!

try ".\replay_analyser.exe"?  or try hitting TAB to tabcomplete it (which should do it for you)  it won't be on your PATH, so you need to address it directly (it's a protection against running things that you don't want to)

 


Edited by Baldrickk, 08 July 2018 - 11:32 AM.


Long_Range_Sniper #32 Posted 08 July 2018 - 11:35 AM

    General

  • Player
  • 30321 battles
  • 8,176
  • [-MM] -MM
  • Member since:
    04-04-2011

View PostBaldrickk, on 08 July 2018 - 10:31 AM, said:

try ".\replay_analyser.exe"?  or try hitting TAB to tabcomplete it (which should do it for you)  it won't be on your PATH, so you need to address it directly (it's a protection against running things that you don't want to)

 

 

Yeah, I went back to your original instructions and spotted the .\ and it was running when you posted this. Churning away now. Cheers.

Jamadeus #33 Posted 08 July 2018 - 11:36 AM

    Corporal

  • Player
  • 17157 battles
  • 157
  • [LOC-X] LOC-X
  • Member since:
    02-19-2015
Thanks, Baldrickk for all the time, knowledge and effort that went into this.

Baldrickk #34 Posted 08 July 2018 - 11:47 AM

    Field Marshal

  • Player
  • 29322 battles
  • 13,387
  • [-TAH-] -TAH-
  • Member since:
    03-03-2013

View PostFlubber42, on 08 July 2018 - 11:15 AM, said:

Hold on, why arent you including LordMuffin in the rating of the friendly team?

 

  • "Position on the Y axis is the average rating of the friendly team (not including LordMuffin himself)"

 

 

The player providing the replays is a constant, they are always in their battles.  The others are (supposedly) random (that's what we want to test)

Lets take a dice game as an example.

Two players, 5 dice.

Each player rolls the 5 dice, the score is the average of all the dice.

This would be like looking at every battle the MM creates in a day.  But we can't see that.

So instead we look at a player's replays (also useful because people claim that they get the MM rigged against them)

This would be like one of those players rolling a dice that only rolls a 6 (or any other number, doesn't matter, point is that it is always the same).  The result is obviously biased now.  We can eliminate the bias by finding the average of the other four dice, ignoring the fixed one.

We do the same here with the MM analysis here.



Enforcer1975 #35 Posted 08 July 2018 - 11:49 AM

    Field Marshal

  • Player
  • 19784 battles
  • 10,270
  • [WJDE] WJDE
  • Member since:
    05-04-2014
I'm too stupid to get this to work mate....i think we need some chat time since i don't really want to upload 7 GB. 

NUKLEAR_SLUG #36 Posted 08 July 2018 - 11:50 AM

    Lieutenant

  • Player
  • 27199 battles
  • 1,825
  • Member since:
    06-13-2015

This thread needs to be the gateway frontpage of the forum with a compulsory 'I have read this' checkbox.

 

 

View PostLong_Range_Sniper, on 08 July 2018 - 11:26 AM, said:

 

:medal:+10 though

 

 

Cheapskate..

 

:medal:+100



Baldrickk #37 Posted 08 July 2018 - 11:54 AM

    Field Marshal

  • Player
  • 29322 battles
  • 13,387
  • [-TAH-] -TAH-
  • Member since:
    03-03-2013

View PostCatn1p, on 08 July 2018 - 04:51 AM, said:

It tells that if you get a streak of terrible teams for 100 games and then a streak of great teams for 100 games then you have gotten average teams for 200 games.

 

Friggin’ genius.

 

But you'd see that on the 5th/6th and especially the 7th graph....

 

And yes, that is how averages work.  Streaks do happen.



Zhul87 #38 Posted 08 July 2018 - 12:53 PM

    Corporal

  • Player
  • 21348 battles
  • 100
  • [TEC] TEC
  • Member since:
    05-31-2013

Block Quote

 

Total replays:

            19854

Green team average rating:

            4622.01

Red team average rating:

            4527.21

Percentage difference:

            +2.07%

Stronger than enemy:

            11039 battles

Weaker than enemy:

            8815 battles

Percentage Stronger:

            55.60%

 

If for LordMuffin the avg. team was 2% stronger for 55% of the time, we have to assume that there are also people out there where the teams are 2% weaker for 55% of the time, because the above can't be true for everyone, right?

Would that qualify to make people feel cheated by the system, if said system unintentionally gives them worse or better odds over a long period of time and large amount of battles, simply due to team composition?

 

Your graph, splitted into two, clearly indicates that the majority of games are won or lost if the difference in team ratings is between +-200 points and that outcome and team rating correlate. So getting a 100 points (dis-)advantage on avg. in 55% of your games seems like a big deal in terms of wr, if the system tries to force all players into 40-60% wr spectrum.

graph_1_defeat.pnggraph_1_win.png

 


Edited by Zhul87, 08 July 2018 - 01:03 PM.


Shuddnobetta #39 Posted 08 July 2018 - 01:01 PM

    Sergeant

  • Player
  • 8726 battles
  • 293
  • Member since:
    11-22-2012
Great post OP and many thanks for the work you put in for the community. Amazing that the results shown so far do not reflect the "general feel" of the game and it's many foibles.

LordMuffin #40 Posted 08 July 2018 - 01:03 PM

    Field Marshal

  • Player
  • 47472 battles
  • 10,573
  • [-GLO-] -GLO-
  • Member since:
    06-21-2011

View PostZhul87, on 08 July 2018 - 12:53 PM, said:

 

If for LordMuffin the avg. team was 2% stronger for 55% of the time, we have to assume that there are also people out there were the teams are 2% weaker for 55% of the time, because the above can't be true for everyone, right?

Would that qualify to make people feel cheated by the system, if said system unintentionally gives them worse or better odds over a long period of time and large amount of battles, simply due to team composition?

 

Your graph, splitted into two, clearly indicates that the majority of games are won or lost if the difference in team ratings is between +-200 points and that outcome and team rating correlate. So getting a 100 points (dis-)advantage on avg. in 55% of your games seems like a big deal in terms of wr, if the system tries to force all players into 40-60% wr spectrum.

graph_1_defeat.pnggraph_1_win.png

 

If the MM tried to get everyone in between 40-60% WR.

Why do I, with above 60% WR still get on average slightly better teams?

Shouldn't I get worse teams?






1 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users