Jump to content
Truthiness

Regionals Data Project 2018-2019 Season

Recommended Posts

16 minutes ago, Cpt. Caine said:

Just to get what you're saying. Are you saying the sample is very small? Because potentially we might be able to collect a petty big sample by my estimation, big enough for power analysis to check out

Sorry. As a scientist, I tend to use colloquialisms a bit freely. In nuclear physics we use the word statistics to mean the number of counts in our data set. But yes, our sample is small, particularly when considering it on the individual, regional basis. It's not meaningful to say Interdictors are overpowered because 100% of Interdictors used were in the top 4 in the Fl regional. Between the diversity of ships and upgrades and the number of fleets we'll actually get, we have a small sample size. Not to say you can't get anything from it, just noting that it is a defining challenge with the analysis.

I'm not by any means trying to stop you, and take what I say with a grain of salt. Feel free to pursue a more robust analysis beyond this.

Share this post


Link to post
Share on other sites
2 hours ago, Astrodar said:

Sorry. As a scientist, I tend to use colloquialisms a bit freely. In nuclear physics we use the word statistics to mean the number of counts in our data set. But yes, our sample is small, particularly when considering it on the individual, regional basis. It's not meaningful to say Interdictors are overpowered because 100% of Interdictors used were in the top 4 in the Fl regional. Between the diversity of ships and upgrades and the number of fleets we'll actually get, we have a small sample size. Not to say you can't get anything from it, just noting that it is a defining challenge with the analysis.

No, but it was conclusions that this analysis in previous years gave us - such as nationwide, Major Rhymer was in 95% of winning Imperial lists at every regional - which led to things like the Rhymer nerf, flotilla nerf, and relay nerf.

If it becomes clear that there is a single thing in common in all winning lists, and it's substantially over-represented in that group vs the overall population, it can help point to an area of the game that needs closer examination for balance.  (It is, FWIW, that comparison of the 'top lists' vs 'overall pool' that is useful.  After all, if Interdictors show up 75% of Imperial lists - you'd expect 3 of the 4 'top 4' lists to have them.  If they were in 75% of overall lists and none of the top-4 had them, that would tell you something itself.  Although if something only shows up in 20% of lists generally, but 100% of the 'top 4'...maybe a problem area.  Figuring out what that margin of error is being the biggest issue given the small data set...which is I guess what you are saying.)

Edited by xanderf

Share this post


Link to post
Share on other sites
32 minutes ago, xanderf said:

(...)

 

3 hours ago, Astrodar said:

(...)

I think we all agree.

But given that analysis of power exists - it isn't like we can't both calculate and be forthcoming with an estimation of how a problematically small sample size might've affected the data and with what likelihood the data might be due to a Type I or Type II error.

It is also likely relevant to simply say 'we don't actually know' about very rare occurrences. Unless the sample is very large, the diversity of lists will definitely be a problem as Astrodar points out. And from an objective point of view, we don't actually KNOW if that is because the things that aren't being taken aren't good - or simply because people THINK they aren't good. That's kind of the entire premise of why a concept such as Sabermetrics has been succesful.

If we were to use a simple Expected Frequency (Chi-square test), we wouldn't need a normally distributed data set and any items appearing at least 5 times across a sample would be beyond the treshold of the test without using Yates correction. Again, I think it all depends on whether we consider each singular Regional a population (with 100% sampling) or if we consider ALL Regionals to be the population and every Regional we have data from the sampling.

In any case, I think it would be prudent to adhere by either the 95% confidence interval when reporting actual metrics OR report standard deviations. Just like when otherwise dealing with statistics ;)

P.s. @Astrodar I've never heard that use of statistics before. 

 

Edited by Cpt. Caine

Share this post


Link to post
Share on other sites

Yeah, I do not have the time nor the expertise to get anywhere close to the rather insane level of statistical analysis you're looking to do. This is very surface level stuff that designed to show the blatantly obvious patterns that need to be addressed. I'm looking for massively disproportionate representation. The main question for this season is whether squadron heavy play has been brought into relative balance. The data last season using the same technique showed a massively disproportionate representation at the top. This season, my hypothesis is that it has been brought down a peg, enough to be considered "balanced." We'll see. I'm also looking closely at the health of true MSU, namely the all small ship fleets that used to work in waves 2-4, but started dropping off around wave 5. In wave 7 they appeared to drop off a cliff. I'm curious if that's a constant. I'm willing to add minor questions at this point, but anything more involves a substantial rebuild. That's not something I'm willing to do in season, even if I were capable of doing so. There's enough work to be done just inputting lists.

Share this post


Link to post
Share on other sites
On 11/30/2018 at 11:22 AM, xanderf said:

Well - let's not overshoot it, then.  Maybe just having a pure count, as the sheet already has for Intel/Rogue/etc, may be sufficient to at least suggest if the question is worth digging further into.

I now have a functioning formula that counts Aces as a percentage of all squadrons taken and I've also managed to make it break it down by placement bracket.

Share this post


Link to post
Share on other sites

@xanderf, @Cpt. Caine

Somewhere lines got crossed and what I was trying to convey was sorely missed.

Summary: Low sample size with meta biases are a challenge when approaching a more intense statistical analysis of this data. For the level at what most want to look at, it's also not really needed. The amount of effort vs payoff may be extremely low. Don't expect Truthiness to do it for you. Feel free to do it. I'm going to myself just for fun.

1 hour ago, Cpt. Caine said:

P.s. @Astrodar I've never heard that use of statistics before. 

That's totally fine. However, it's not particularly uncommon. You can read up on its use in this manner here: https://en.wikipedia.org/wiki/Statistic

Share this post


Link to post
Share on other sites
1 hour ago, Truthiness said:

I'm also looking closely at the health of true MSU, namely the all small ship fleets that used to work in waves 2-4, but started dropping off around wave 5. In wave 7 they appeared to drop off a cliff. I'm curious if that's a constant.

Did Vader Cymooon kill it, or Strat Ad, or a little of A and B? If you don't have a plan against it, you're gonna have a bad time.

Share this post


Link to post
Share on other sites
18 minutes ago, geek19 said:

Did Vader Cymooon kill it, or Strat Ad, or a little of A and B? If you don't have a plan against it, you're gonna have a bad time.

Strat Adviser I suspect is the main culprit. Small ships are still being used. They're just almost always being used alongside a large.

Edited by Truthiness

Share this post


Link to post
Share on other sites
1 hour ago, Truthiness said:

I now have a functioning formula that counts Aces as a percentage of all squadrons taken and I've also managed to make it break it down by placement bracket.

Thanks - I think, if it does prove to be true that squadrons are still over-represented at the top - that it will be helpful to have exactly this information before the nerf bat is revisited on them, again!  True generic squadron lists really took such a beating in the last round of nerfing, and I can't help but feel that this is possibly at the fault of aces being too powerful and that messing up overall analysis of squadron play value.  (IE., my theory that it isn't squadrons per se that are too powerful, just that nearly all the aces are woefully under-priced)

Share this post


Link to post
Share on other sites
15 minutes ago, Truthiness said:

Strat Adviser I suspect is the main culprit. Small ships are still being used. They're just almost always being used alongside a large.

The more I think about it, the more I agree with you. Larges now cost half as much for an activation, and with Gunnery Team/ECM so easily available on most of those larges... Also why mediums are being pushed out of the game even faster I think.

Share this post


Link to post
Share on other sites
4 hours ago, geek19 said:

The more I think about it, the more I agree with you. Larges now cost half as much for an activation, and with Gunnery Team/ECM so easily available on most of those larges... Also why mediums are being pushed out of the game even faster I think.

Because if you don't have a points fortress, how do you win a game with a large enough MoV to advance?

Share this post


Link to post
Share on other sites
7 hours ago, Truthiness said:

Yeah, I do not have the time nor the expertise to get anywhere close to the rather insane level of statistical analysis you're looking to do. This is very surface level stuff that designed to show the blatantly obvious patterns that need to be addressed. I'm looking for massively disproportionate representation (...)

As @Astrodar pointed out, no one is expecting you to do the data analysis all by yourself. The entire point here was us offering help.

I'm not sure why doing very basic statistical analysis rather than 'just' looking at descriptives and considering a simple power analysis is insane though? I agree that massively disproportinate trends SHOULD be visible to the naked eye but you do risk Type I errors (false positivies) that way with smaller sample sizes and you might miss significant differences still (Type II). Doing analysis (if the data is there) is the difference between being 'somewhat sure' that there in reality is a tendency in the Armada community and being about 95% sure and thus I don't see how it is insane.

I will mention that doing an expected frequency analysis on select items is a formular readily available in excel and power analysis could easily be done for a few treshold values using a calculator (G*Power is the gold standard and it's free).

7 hours ago, Astrodar said:

@xanderf, @Cpt. Caine

Somewhere lines got crossed and what I was trying to convey was sorely missed.

Summary: Low sample size with meta biases are a challenge when approaching a more intense statistical analysis of this data. For the level at what most want to look at, it's also not really needed. The amount of effort vs payoff may be extremely low. Don't expect Truthiness to do it for you. Feel free to do it. I'm going to myself just for fun.

That's totally fine. However, it's not particularly uncommon. You can read up on its use in this manner here: https://en.wikipedia.org/wiki/Statistic

That's litterally what I saw @xanderf say and myself as well ;) So I don't think there's any misunderstandings here.

Interesting. Well. Uncommon enough in the social sciences for me to never have heard of it. But that's not surprising given the different cultures across fields, and completely different applications and objects of statistics at that. I'm curious to see if your background would lead you to apply dramatically different tools of analysis than I would

Edited by Cpt. Caine
Typo.

Share this post


Link to post
Share on other sites

Here we go; the lists from the first Regional in Queensland, Australia. 11 players, 3 rounds, no cut. Some quick stats:

Factions: 5 Imperial, 6 Rebel

Admirals: Thrawn (2), Moff Jerry (1), Ozzel (2), Rieekan (1), Sato (1), Raddus (2), Ackbar (1), Leia (1)

Average Points on Squadrons: 47.2 points

 

1st place, 3/0, 30 points, 1087 MoV

378 points

Objectives: Advanced Gunnery, Contested Outpost, Solar Corona

ISD Kuat Refit + Grand Admiral Thrawn + Avenger + Strategic Adviser + Boarding Troopers + Electronic Countermeasures + Leading Shots + External Racks [170]

Gladiator I + Demolisher + Captain Brunson + Ordnance Experts + Engine Techs + Assault Proton Torpedoes [88]

Raider I + Darth Vader + External Racks [50]

Gozanti Cruisers + Suppressor + Minister Tua + Electronic Countermeasures + Slicer Tools [43]

Gozanti Cruisers + Hondo Ohnaka + Comms Net [27]

0 squadrons

 

2nd Place, 3/0, 23 points, 544 MoV

400 points

Objectives: Advanced Gunnery, Fire Lanes, Sensor Net

Interdictor Supression + Moff Jerry + Captain Brunson + Fighter Coordination Team + Targeting Scramblers + Grav Shift Reroute + Interdictor [131]

ISD Kuat + Strategic Adviser + Boarding Troopers + Early Warning System + Leading Shots + External Racks + Avenger [138]

Raider I + Ordnance Experts + External Racks [51]

Gozanti Cruisers + Comms Net [25]

Gozanti Cruisers + Comms Net [25]

SQUADRONS: Lambda Shuttle x2 [30]

 

3rd Place, 2/1, 22 points, 423 MoV

390 points

Objectives: Precision Strike, Contested Outpost, Superior Positions

MC30c Torpedo Frigate + Lando Calrissian + Ordnance Experts + External Racks + Admonition [82]

Hammerhead Torpedo Corvette + General Rieekan [66]

Nebulon-B Escort Frigate + Flight Commander + Fighter Coordination Team + Yavaris [68]

GR-75 Medium Transports + Adar Tallon + Bomber Commander Center + Boosted Comms [40]

GR-75 Medium Transports + Toryn Farr + Bright Hope [27]

SQUADRONS: Wedge Antilles [19], Biggs Darklighter [19], Ten Nunb [19], Dagger Squadron [15], Dutch Vander [16], Jan Ors [19]

 

4th Place, 2/1, 18 points, 388 MoV

389 points

Objectives: Advanced Gunnery, Planetary Ion Cannon, Superior Positions

MC-75 Ordnance + Commander Sato + Strategic Adviser + Ordnance Experts + Electronic Countermeasures + Assault Proton Torpedoes [152]

MC30c Scout Frigate + Intel Officer + Ordnance Experts + Assault Proton Torpedoes + XI7 Turbolasers + Foresight [99]

GR-75 Medium Transports + Toryn Farr + Bomber Command Center + Expanded Hangar Bay + Bright Hope [40]

SQUADRONS: Nym [21], Jan Ors [19], Scurrg x2 [32], X-wing x2 [26]

 

5th Place, 2/1, 17 points, 312 MoV

395 points

Objectives: Advanced Gunnery, Capture the VIP, Solar Corona

MC80 Assault Cruiser + Admiral Raddus + Hondo Ohnaka + Electronic Countermeasures + Early Warning System + Quad Battery Turrets + Leading Shots + Engine Techs + Home One

MC75 Ordnance Cruiser + Damage Control Officer + Ordnance Experts + Electronic Countermeasures + Expanded Launchers + Assault Proton Torpedoes + Profundity

Hammerhead Torpedo Corvette + Cham Syndulla + External Racks + Garel’s Honour

GR-75 Medium Transports + Slicer Tools + Quantum Storm

 

6th Place, 1/2, 14 points, 140 MoV

394 points

Objectives: Opening Salvo, Planetary Ion Cannon, Solar Corona

Assault Frigate MkIIB + Admiral Ackbar + Raymus Antilles + Caitken & Sholla + Electronic Countermeasures + XI7 Turbolasers [136]

CR90A + Intel Officer + Turbolaser Reroute Circuits + Jaina’s Light [60]

CR90A + Intel Officer + Turbolaser Reroute Circuits [58]

CR90A + Intel Officer + Turbolaser Reroute Circuits [58]

GR-75 Medium Transport + Bright Hope + Leia Organa [23]

GR-75 Medium Transport + Quantum Storm + Slicer Tools [26]

SQUADRONS: Tycho Celchu [16], Shara Bey [17]

 

7th Place, 1/2, 13 points, 167 MoV

400 points

Objectives: Most Wanted, Contested Outpost, Salvage Run

ISD Kuat + Grand Admiral Thrawn + Governor Pryce + Ruthless Strategists + Electronic Countermeasures + External Racks + Leading Shots [169]

Gladiator II + Captain Brunson + Ruthless Strategists + External Racks + Demolisher [84]

Gozanti Cruisers + Comms [25]

Gozanti Cruisers + Comms [25]

SQUADRONS: Dengar [20], TIE Bomber x4 [36], Mauler Mithel [15], Lambda Shuttle [15], TIE Interceptor [11]

 

8th Place, 1/2, 12 points, 155 MoV

390 points

Objectives: Most Wanted, Hyperspace Assault, Dangerous Territory

ISD Cymoon + Admiral Ozzel + Minister Tua + Electronic Countermeasures + Heavy Turbolaser Turrets + XI7 Turbolasers + Relentless + Q7 Tractor Beams [162]

ISD 1 + Q7 Tractor Beams + leading Shots + XX8 Turbolasers + Avenger [130]

Raider I + Agent Kallus + Ordnance Experts + Expander Hangar Bay + Rapid Reload [68]

SQUADRONS: Ciena Ree [17], Valen Ruder [13]

 

9th Place, 1/2, 12 points, 116 MoV

399 points

Objectives: Opening Salvo, Contested Outpost, Minefields

MC80 Assault + Admiral Raddus + Strategic Adviser + Advanced Projectors + Electronic Countermeasures + XI7 Turbolasers + Defiance [168]

MC75 Ordnance Cruiser + Ordnance Experts + Electronic Countermeasures + External Racks + Assault Proton Torpedoes [119]

Nebulon-B Support Refit + Dual Turbolaser Turrets + Salvation [63]

GR-75 Medium Transports + Comms Net + Quantum Storm [21]

 

10th Place, 1/1 (drop), 11 points, 400 MoV

379 points

Objectives: Most Wanted, Fighter Ambush, Superior Positions

Raider I + Admiral Ozzel [64]

Gladiator I + Minister Tua + Electronic Countermeasures + Ordnance Experts + Engine Techs + Assault Proton Torpedoes + Demolisher [92]

Gozanti Cruisers [23]

Gozanti Cruisers + Hondo [25]

SQUADRONS: Dengar [20], Major Rhymer [16], VT-49 Decimator x4 [88]

 

11th Place, 0/3, 9 points, 0 MoV

Objectives: Opening Salvo, Planetary Ion Cannon, Dangerous Territory

Assault Frigate MkIIA + Walex Blissex + Caitken & Sholla + Electronic Turbolasers + Spinal Armaments + Paragon [113]

CR90B + Leia Organa (admiral) + SW7 Ion Batteries [82]

CR90B + Engine Techs + High Capacity Ion Turbines + Dodonna’s Pride [61]

Pelta Assault + Raymus Antilles + Intensify Firepower + Disposable Capacitors [76]

GR-75 Combat Refit + Slicer Tools [31]

GR-75 Combat Refit + Slicer Tools [31]

Edited by DR4CO

Share this post


Link to post
Share on other sites
4 hours ago, Ginkapo said:

@Truthiness Its a piece of piss to do reliability checks on your data without you doing anything different. Generally though I dont bother as its blatantly statisrically unreliable due to sample size. (And we all know that)

Two tail tests are inappropriate. 

You're speaking gibberish to me :D

Look, anybody is free to take this stuff and apply whatever statistical model they like. I just encourage you to ask yourself why. You're talking about a ton of effort for what I think is very little productive value. Feel free to prove me wrong. I just don't like to see someone spin their wheels for little gain.

Edited by Truthiness

Share this post


Link to post
Share on other sites
11 minutes ago, CaribbeanNinja said:

CaribbeanNinja Regionals Data Analysis:

Empire: 3

Rebels: 0

100% of winners are Empire faction. 

0% of winners are Rebels. 

Have you got confidence intervals for these statistics? 

Share this post


Link to post
Share on other sites

Metrics that I have used in the past are:

Top % - bottom % : tells me if a card is biased top/bottom

Top% + bottom%: how skill dependent a card is

Both of these could do with a representation weighting.  The only person who brought Hera winning a regional is less significant than  if 27 people brought Hera and all ended up in the top bracket.

% of ship points spent on a specific ship variant:. Tells me which ships are being taken surprisingly often or rarely.  Every rebel fleet taking an mc80 is more significant than every rebel fleet taking a cr90.

Share this post


Link to post
Share on other sites
Quote

9th Place, 1/2, 12 points, 116 MoV

399 points

Objectives: Opening Salvo, Contested Outpost, Minefields

MC80 Assault + Admiral Raddus + Strategic Adviser + Advanced Projectors + Electronic Countermeasures + XI7 Turbolasers + Defiance [168]

MC75 Ordnance Cruiser + Ordnance Experts + Electronic Countermeasures + External Racks + Assault Proton Torpedoes [119]

Nebulon-B Support Refit + Dual Turbolaser Turrets + Salvation [63]

GR-75 Medium Transports + Comms Net + Quantum Storm [21]

This list is missing 28 points somewhere. I've put in Shara and an A-Wing to fill the gap.

Share this post


Link to post
Share on other sites
8 hours ago, DR4CO said:

10th Place, 1/1 (drop), 11 points, 400 MoV

379 points

Objectives: Most Wanted, Fighter Ambush, Superior Positions

Raider I + Admiral Ozzel [64]

Gladiator I + Minister Tua + Electronic Countermeasures + Ordnance Experts + Engine Techs + Assault Proton Torpedoes + Demolisher [92]

Gozanti Cruisers [23]

Gozanti Cruisers + Hondo [25]

SQUADRONS: Dengar [20], Major Rhymer [16], VT-49 Decimator x4 [88]

This is missing 51 points. I've assumed it is a Raider I with OE and ER since it fits perfectly.

Share this post


Link to post
Share on other sites

@DR4CO Thanks for pulling that together. I only pointed out the errors so that everyone could see the alterations I was making for the data sheet. @Vath deserves a shout out for getting the KC lists together as well!

Now if only my favorite Georgia Peach would send me Orlando's lists =D

Share this post


Link to post
Share on other sites
21 minutes ago, Truthiness said:

@DR4CO Thanks for pulling that together. I only pointed out the errors so that everyone could see the alterations I was making for the data sheet. @Vath deserves a shout out for getting the KC lists together as well!

Now if only my favorite Georgia Peach would send me Orlando's lists =D

Sweet Georgia Paelleon? Giled on my mind? 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...