Jump to content
Truthiness

Regionals Data Project 2018-2019 Season

Recommended Posts

13 hours ago, Ginkapo said:

@Truthiness Its a piece of piss to do reliability checks on your data without you doing anything different. Generally though I dont bother as its blatantly statisrically unreliable due to sample size. (And we all know that)

(...)

Whether or not it is statistically sound to draw conclusions is always based on sample size. I don't understand the incessant insistence that it is impossible to do actual analysis instead of guesstimates. I fail to understand the point of doing data collection at all if one does not intend to actually find out what the data means, rather than just looking at averages and comparing averages and then guessing based on that without ever checking to see if one's guess was correct. I don't how many lists previous analysis were based on. I'm curious to hear the number and obviously a sample size that is too small could put a dampener on my hopes for aiding in the developing Landometrics for Armada. Based on what has been promised in this thread already, though, it looks as if getting to 300ish fleet lists isn't outside the realm of possibility.

At that point, all relevant data points (under the assumption that upgrades, ships, commanders and objectives that are taken almost never aren't relevant for an analysis of efficacy in the current meta anyway) should be made up of high enough frequencies to not violate the assumptions of most categorical data test statistics. Goodness of Fit and other Chi-Square metrics only require no less than 5 observations of an item to be valid, and there are various corrections (I mentioned Yates previously, but upon reading up on it Fisher's Exact test would be a better idea) to account even for that.

If by 'reliability checks' you mean power analysis, I'm not sure what you mean by that. It is, after all, a calculation that EVALUATES your sample size - not the other way around.

You also have to consider the fact that the POPULATION in this case is extremely small.

13 hours ago, Ginkapo said:

(...)

Two tail tests are inappropriate. 

I completely agree. I can't come up with a situation where the raw data wouldn't suggest a one-tailed test. I haven't heard anyone suggest two-tailed tests in the thread. All the test statistics I've mentioned are one-tailed by design.

8 hours ago, Truthiness said:

(Directed at Ginkapo) You're speaking gibberish to me :D 

Look, anybody is free to take this stuff and apply whatever statistical model they like. I just encourage you to ask yourself why. You're talking about a ton of effort for what I think is very little productive value. Feel free to prove me wrong. I just don't like to see someone spin their wheels for little gain.

Thank you. I look forward to digging into the data and in helping you gather it!

But please, understand that I - at least - am not trying to tell you to either a) do more work, b) do work for 'little productive value', nor b) simply do more rigorous analysis for the **** of it. What I'm trying to do is add value to YOUR work, without you doing anything. I'm sure what @Astrodar was saying he'd look into and what I'd like to help with is both intended for broad dissemination in conjunction with whatever you're attempting to do for Steelstrategy.com.

My point with my original post in this thread was to start a discussion so that we could all collaborate on developing better metrics and more smart and insightful analysis. Just like a gaming community ought to do, I'd think. Especially when people are making the effort to collect the data and write it up. I'm entirely unsure why that seems to be perceived as hostile or arrogant by some participants here.

Anyway. Based on the discussion in the thread (and your remark to Ginkapo regarding some statistical terms), it appears to me that you might not realise the finer points of the difference between comparing descriptives and actually running an analysis (whether that analysis is super simple or not, the data largely determined that by itself anyway).

Let me make my case as to why the difference between comparing descriptives and running an analysis isn't the difference between getting it 80% right or 85% right, but the difference between not even knowing if what you're looking at is even likely to be representative and being up to 95% sure you're right.

You are - I believe - also from the intelligence community, so I'm sure you've read Thinking Fast and Slow, yes?

The mathematical reasons for relying on analysis rather than naked-eye, descriptive based conclusions are well, self-evident, but also unconvincing to many. Put simply, any observation has a certain probability to be random - so if you draw conclusions without accounting for the randomness and the various intricacies of sampling, you're very likely to make Type I (erroneously confirming your hypothesis to be true) and Type II errors (erroneously finding your hypothesis to be wrong). However. The psychological, intuitive reason as to why simply guesstimating based on descriptives is bound to leave you about exactly as proficient as a dart-throwing chimpanzee (the favourite example in popular science, feel free to substitute another image) - i.e. chance level - in being right is that humans are rather inept innate mathematicians and extremely prone to drawing conclusions when they shouldn't. This is especially true when it comes to pattern analysis -such as... tendencies in data on Armada games - especially if they patterns are expected, but also when they're unexpected.

Example.

If you look at a dataset of fleet lists with a total sample size of 320. And find that there are

  • 176¬†Imperial players (55%), of which 62 (35.2%) brought Sloane and 29 (16.5%) Jerjerrod as a commander.
  • 144 Rebel players (45%), of which 24 (16.7%) brought Raddus and 45 (31.3%) Ackbar as a commander.
  • 35 Imperial players (54.7%) advanced to the top 4 - 15 with Sloane (42.9%), 6 with JJ (17.1%)
  • 29 Rebel players (45.3%) advanced to the top 4 - 6 with Raddus (6.7%) and 5 with Ackbar (17.2%)

You could conclude that an increase of 21.9% in Sloane appearance in the top 4 among Imperials and a decrease of 45% in regards to Ackbar is mildly interesting in the first case and worth considering important in the latter case. If you run the numbers through a Goodness of Fit test, however, you'll end up with insignificant X^2 values. I ran Goodness of Fit tests with an alpha value of .05.

Testing whether the number of Ackbar fleets among the top 4, given the base rate of Ackbar seen among all fleets, is significantly different from what could be expected (i.e. are top 4 players consistently using Ackbar more than other players, which would suggest that Ackbar is a part of the reason or alternatively, that they're better at using him, or a combination) you'll reach  X^2= 2.58 (p=.11) - and in the same case with Sloane X^2=1.38 (p=.2).

I also did a power analysis, primarily to prove that it IS high due to the nature of chi-square statistics:

I ended up in the range of .79 with the Ackbar case (given an effect size that is still ok large) and around the .7 mark with the Sloane case. Which would lead me to conclude that even given smaller data sets, the power level would be similar to many respectable scientific studies. So not something to fret too much about.

-----

Given the above (and intricacies and controversies in the field of Statistics) I'd wager the following:

  • If you can gather data from 12-16 Regionals, you will have a data set robust enough to be statistically sound in general - and in practice good enough to detect common upgrades and fleet composition data points¬†across factions (since the sample is by definition about twice the size when it isn't faction specific), as well as very common ships, upgrades and commanders that are faction specific.
  • If you can gather data from upwards of 24 Regionals or more, I don't really see any relevant sample size issues - if we, again, assume that things that aren't present in Regional fleets (or almost not present) are irrelevant.

And, further, recommend:

  • Running with a .1 significance treshold and generally not considering power analysis, except to dip a toe when it comes to finding the treshold of where the less common items become untenable to analyse. That'll yield exactly¬†10% chance of erroneously concluding that the top 4 IS taking something more frequently in reality than others and about a maximum 30% chance and as less as 10% on average¬†of not finding a difference when it is there. If that sounds like a lot of chance to be wrong, let me just remind you that doing raw data analysis¬†leaves you at about a 50% chance of being wrong¬†either way.
  • Selectively use goodness of fit (I can't find a reason why the CHISQ.TEST formula in Excel shouldn't be okay to use, it isn't EXACTLY goodness of fit, but it is close) on large differences to test for it. It'll take you either three click in excel (easiest) or 35 seconds googling 'Goodness of Fit calculator' and putting in the four data points needed for a calculation.

 

-----

As previously mentioned, I'll gladly help. But if you want to improve your data analysis from fortune telling to actual analysis - taking these simple steps will do that.

Edited by Cpt. Caine

Share this post


Link to post
Share on other sites
29 minutes ago, Ginkapo said:

So it was completely pointless to run reliability analysis? I told you that without doing a single piece of arithmatic. 

You don't actually know what 'reliability analysis' is, do you?

I'm going to assume you think power analysis is a measure of reliability.

No. It was not pointless. And it would be oxymoronic to say it is, whilst holding that the sample size is too small to run robust analysis. Like you did in your last post thqt i quoted earlier. Those two realities can not coexist.

I just proved that the power is adequate, and by extension the sample size is, for faction specific analysis on high frequencies at the level of around 15 Regionals collected.

Edited by Cpt. Caine

Share this post


Link to post
Share on other sites
3 hours ago, Truthiness said:

Got some additional info from the KC Regional. All lists are there now, short some objectives. Thanks again to @Vath

No problem! Sorry I couldn’t grab those objectives. It bothers me seeing unfinished data. 

 

Also, @Cpt. Caine could you please move on from whatever you’re discussing so we can keep this thread specifically about collecting the regional data for Truthiness?

Edited by Vath

Share this post


Link to post
Share on other sites
10 hours ago, Truthiness said:

Four more Regionals this weekend. Good luck to everyone playing. Bring back data for the data god!ÔĽŅ

Portland, OR

San Antonio, TX

Newington, CT

Quebec, Canada

I'll be at the Regional in Quebec, I'll try to bring back as much data as I can.

Such as what everyone had for breakfast and lunch, how many steps they took during the day, the average ratio of men:ladies getting their hair cut (it's taking place in front of a hair salon). That's the data you need, right?

Share this post


Link to post
Share on other sites

Here's the Regional Data for Quebec City. Posting here in case it's of any interest/inspiration to anyone.

We had an amazing 20 players (we average 8-10 during Store Champs), from all over the province. We even had 3 guys who came down from Toronto, including the Canadian National Champion. 

Here's the data. I have all 20 lists. There may be 1 or 2 inaccuracies as for some I had only a picture of the fleet on the table. Likewise, for the same reason, I don't have all the objectives for everyone. Also compiled some lightweight stats at the top.

( @Truthiness, let me know if you spot any mistakes)

Avg Bid: 9.1 (7x 0 point, 5x >=20 points)

Avg Squadrons: 4.95 squadrons, 75.45 points, (5x squadronless, 10x >=100 points)

Avg Number of deployments: 6.45

Avg Number of activations: 4.1 ( 9x 3 activations, highest: 8 )
 

Admirals

Rebel (9 players): Raddus (2), Sato (2), Ackbar (1), Dodonna (1), Garm (1), Mothma (1), Rieekan (1)

Imperial (11 players): Motti (3), Jerjerrod (2), Sloane (2), Thrawn (2), Vader (2)
 

Ships

Rebels: GR75 Medium Transport (11), Hammerhead Torpedo Corvette (6), CR90B (6), MC30c Torpedo Frigate (6), MC75 Ordnance Cruiser (3), CR90A (2), GR75 Combat Retrofit (2), Nebulon-B Escort Frigate (2), Pelta Assault Frigate(2), MC75 Armored Cruiser (1), MC80 Battle Cruiser (1), LMC80 Star Cruiser (1)

Imperial: Gladiator I (10), Gozanti Cruisers (10), Cymoon (5), Gladiator II (3), ISD II (2), Victory II (2), Quasar II (2), Arquitens (1), ISD I (1), Kuat (1), Victory I (1)

 

1st Place - Mac - 28 points - 940 MoV

400 points

ISD II - Admiral Sloane, Strategic Adviser, Hardened Bulkheads, Early Warning System, Leading Shots, QBT, Avenger

Gladiator I - Captain  Brunson, Ordnance Experts, Engine Techs, APT, Demolisher

Gozanti Cruisers - Minister Tua, Jamming Field, Suppressor, ECM

Squadrons (100 points) - Maarek Stele, Mauler Mithel, Colonel Jendon, Jumpmaster 5000, 4x TIE Fighters

 

2nd - Sebastien D - 26 points - 781 MoV

380 points

ISD Cymoon Refit - Darth Vader, Captain Brunson, Gunnery Team, QTC, H9 Turbolasers, QLT, Entrapment Formation

ISD Cymoon Refit - Strategic Adviser, Gunnery Team, XI7, H9, QLT, Intensify Firepower!

Gozanti Cruiser - Comms Net

Gozanti Cruiser - Comms Net

 

3rd - Louis-André - 25 points - 682 MoV

397 points

Objectives - Most Wanted, Contested Outpost, Solar Corona

Pelta Assault Ship - Admiral Ackbar, Entrapment Formation!, External Racks

Pelta Assault Ship - Intensify Firepower!, External Racks

CR90B - Heavy Ion Emplacements

CR90B - Heavy Ion Emplacements

CR90B - Heavy Ion Emplacements

CR90B

GR-75 Medium Transport - Toryn Farr, Bright Hope

GR-75 Medium Transport - Comms Net

 

4th - Kristjan - 24 points - 551 MoV

398 points

Victory I - Admiral Motti, Minister Tua, Ordnance Experts, QBT, External Racks, ECM, Warlord

Gladiator I - Captain Brunson, Ordnance Expert, APT, Demolisher

Gladiator II - Agent Kallus, Ruthless Strategist, External Racks

Squadrons (120 points) - Morna Kee, Captain Jonus, Colonel Jendon, Maarek Stele, 2x Firespray-31

 

5th - Jean-Michel (aka @Sybreed) - 19 points - 468 MoV

391 points

Objectives - Opening Salvo, Hyperspace Assault, Solar Corona

CR90A - Mon Mothma, TRC, Jaina’s Light

CR90B - Heavy Ion Emplacements

MC30c Torpedo Frigate - Lando Calrission, Ordnance Experts, APT, Admonition

MC30c Torpedo Frigate - Ordnance Experts, APT, Foresight

Hammerhead Torpedo Frigate - External Racks, Garel’s Honor

GR75 Medium Transport - Comms Net

Squadrons (33 points) - Shara Bey, Tycho Celchu

 

6th - Sebastien Légaré (aka @hratly) - 19 points - 412 MoV

390 points

Objectives - Precision Strike, Planetary Ion Cannon, Superior Positions

ISD I - Grand Admiral Thrawn, Avenger, Damage Control Officer, Boarding Troopers, Dual Turbolaser Turrets, Leading Shots

Gladiator I - Demolisher, Captain Brunson, Ordnance Experts, Engine Techs, ACM

Gozanti Cruisers - Hondo Ohnaka, BCC

Squadrons (103 points) - Dengar, Mauler Mithel, Tempest Squadron, TIE Advanced, TIE Defender, 3x TIE Bomber

 

7th - Carlo - 18 points - 436 MoV

379 points

CR90B - Admiral Raddus, Lando Calrissian, Engine Techs, MS-1 Ion Cannons, Jaina’s Light

MC75 Ordnance Cruiser - Intel Officer, Ordnance Experts, Hardened Bulkheads, ECM, ACM, Expanded Launchers, Aspiration

GR75 Medium Transport - Hondo Ohnaka, Quantum Storm

Squadrons (131 points) - Jan Ors, 7x YT-2400

 

8th - Mario - 18 points - 188 MoV

380 points

MC75 Ordnance Cruiser - Commander Sato, Engineering Captain, Ordnance Experts, ECM, External Racks, Ordnance Pods, Profundity

Hammerhead Torpedo Corvette -  Ordnance Experts, External Racks, Garel’s Honor

GR-75 Combat Retrofit - Repair Crews

GR-75 Combat Retrofit - Repair Crews

Squadrons (115 points) - Tycho Celchu, Jan Ors, 5x YT-2400

 

9th - Olivier - 17 points - 293 MoV

400 points

Objectives - Blockade Run, Contested Outpost, Sensor Net

ISD Cymoon - Admiral Motti, Strategic Adviser, Entrapment Formation!, Gunnery Team

ISD Kuat - Agent Kallus, Gunnery Team, Reinforced Blast Doors, Leading Shots, External Racks

Gozanti Cruiser - Comms Net

Squadrons (89 points) - Tempest Squadron, Mauler Mithel, Jumpmaster 5000, TIE Advanced, VT-49 Decimator, Lambda Shuttle

 

10th - Serge - 17 points - 105 MoV

387 points

Objectives - Opening Salvo, Contested Outpost, Dangerous Territory

ISD Cymoon - Admiral Motti, Captain Needa, Entrapment Formation, Veteran Gunners, TRC, XI7

Gladiator I - Captain Brunson, Ordnance Experts, APT, Demolisher

Gladiator I - Agent Kallus, Ordnance Experts, APT

Gozanti Cruisers - Comms Net

Squadrons (53 points) - Maarek Stele, 2x TIE Defender

 

11th - Maxim (aka @AdmiralOldOwlz)- 16 points - 221 MoV

382 points

Objectives - Precision Strike, Fighter Ambush, Superior Positions

Victory II - Grand Admiral Thrawn, Gunnery Team, Boosted Comms, QBT, Leading Shots

Gladiator II - Demolisher, Ordnance Experts, APT

Gozanti Cruisers - Boosted Comms, BCC

Squadrons (129 points) - Tempest Squadron, Maarek Steele, Dengar, Zertik Strom, Mauler Mithel, 5x TIE Bomber

 

12th - Samuel - 16 points - 5 MoV

380 points

Quasar II - Moff Jerjerrod, Captain Brunson, Ruthless Strategists, Boosted Comms

ISD II - Governor Pryce, Gunnery Team, EWS, Leading Shots, XI7

Gozanti Cruisers - BCC

Squadrons (101 points) - Major Rhymer, Mauler Mithel, Tempest Squadron, Jumpmaster 5000, 5x TIE Bomber

 

13th - Sebastien Ladouceur - 15 points - 253 MoV

374 points

ISD Cymoon - Darth Vader, Agent Kallus, Gunnery Team, QLT, XI7, QBT

Gladiator II - Demolisher, Captain Brunson, Ordnance Experts, Engine Techs, APT

Gozanti Cruiser - Jamming Field

Squadrons (90 points) - Soontir Fel, Mauler Mithel, Howlrunner, Dengar, TIE Advanced

 

14th - Francis - 15 points - 195 MoV

400 points

MC30c Torpedo Frigate - Admiral Raddus, Lando Calrissian, Ordnance Experts, External Racks, Admonition

MC75 Armored Cruiser - Caitken and Shollan, ECM, External Racks, HIE, XI7, Profundity

CR90A - Engine Techs, TRC, Jaina’s Light

Hammerhead Torpedo Corvette - Ordnance Experts, External Racks, Garel’s Honor

GR75 Medium Transport - Comms Net, Bright Hope

GR75 Medium Transport - Hondo Ohnaka

 

15th - Simon G (aka myself) - 13 points - 227 MoV

400 points

Objectives - Opening Salvo, Planetary Ion Cannon, Sensor Net

MC75 Ordnance Cruiser - Commander Sato, Lando Calrissian, Ordnance Experts, ECM, External Racks, ACM

Hammerhead Torpedo Corvette - Ordnance Experts, Disposable Capacitors, ACM, TFO

Hammerhead Torpedo Corvette - Ordnance Experts, Disposable Capacitors, ACM, TFO

Hammerhead Torpedo Corvette - Ordnance Experts, Disposable Capacitors, ACM, TFO

GR75 Medium Transport - Jamming Field

Squadrons (70 points) - Tycho Celchu, 2x HWK-290, 2x VCX-100

 

16th - Simon M - 13 points - 138 MoV

400 points

Arquitens Light Cruiser - Admiral Sloane, Captain Needa, TRC

Gladiator I - Demolisher, Ordnance Experts, APT

Quasar II - Agent Kallus, Flight Controllers, Boosted Comms

Gozanti Cruisers - Suppressor, Instructor Goran

Squadrons (130 points) - Dengar, Zertik Strom, Valen Rudor, Darth Vader, Saber Squadron, Howlrunner, Mauler Mithel, Soontir Fel

 

17th - Danny - 13 points - 27 MoV

400 points

MC80 Battle Cruiser - General Rieekan, Intel Officer, XI7, Mon Karren

Nebulon-B Escort Frigate - Toryn Farr, Yavaris

GR75 Medium Transport - Adar Tallon, BCC

GR75 Medium Transport - Raymus Antilles, Comms Net

Squadrons (114 points) - Nym, Norra Wexley, Jan Ors, Wedge Antilles, Gold Squadron, Rogue Squadron, Green Squadron

 

18th - Robert - 12 points - 320 MoV

399 points

Objectives - Most Wanted, Hyperspace Assault, Dangerous Territory

Victory I - Moff Jerjerrod, Intel Officer, Ordnance Experts, XI7 Turbolasers, External Racks

Gladiator I - Intel Officer, Ordnance Experts, External Racks, Demolisher

Gladiator I - Intel Officer, Ordnance Experts, External Racks

Gladiator I - Intel Officer, Ordnance Experts, External Racks

Gladiator I - Intel Officer, Ordnance Experts, External Racks

 

19th - Carl (aka @At0micTurtle) - 12 points - 218 MoV

400 points

Objectives - Most Wanted, Fleet Ambush, Superior Positions

MC80 Star Cruiser - Garm Bel Iblis, Adar Tallon, Caitken and Shollan, HIE, QBT, Liberty

Nebulon-B Escort Frigate - Flight Commanders, Fighter Coordination Team, Yavaris

GR75 Medium Transport - Ahsoka Tano

GR75 Medium Transport - Toryn Farr, Bright Hope

Squadrons (131 points) - Biggs Darklighter, Gold Squadron, Dutch Vander , Jan Ors, Ten Nunb, Wedge Antilles, B-Wing, X-Wing

 

20th - Martin - 4 points - 0 MoV

385 points

MC75 Ordnance Cruiser - General Dodonna, Ordnance Experts, EWS, Ordnance Pods, APT

MC30c Torpedo Frigate - Ordnance Experts, Advanced Projectors, External Racks, XI7

MC30c Torpedo Frigate - Ordnance Experts, Advanced Projectors, External Racks, XI7

MC30c Torpedo Frigate - Ordnance Experts, Advanced Projectors, External Racks, XI7

Edited by CptAwesomer

Share this post


Link to post
Share on other sites
17 minutes ago, CptAwesomer said:

3rd - Louis-André - 25 points - 682 MoV

397 points

Objectives - Most Wanted, Contested Outpost, Solar Corona

Pelta Assault Ship - Admiral Ackbar, Entrapment Formation!, External Racks

Pelta Assault Ship - Intensify Firepower!, External Racks

CR90B - Heavy Ion Emplacements

CR90B - Heavy Ion Emplacements

CR90B - Heavy Ion Emplacements

CR90B

GR-75 Medium Transport - Toryn Farr, Bright Hope

GR-75 Medium Transport - Comms Net

Awesome list! I'd be intrigued to try this.

Share this post


Link to post
Share on other sites
11 minutes ago, Sybreed said:

trust me, it's very brutal...

I'm a little confused on the selection of ackbar, as you get more totally dice if you just double arch stuff, I guess he is just insurance for the times you only have a side shot?

Share this post


Link to post
Share on other sites
9 hours ago, CptAwesomer said:

4thÔĽŅ - ÔĽŅKristian - 24 points - 551 MoV

394 points

Victory I - Admiral Motti, Minister Tua, QBT, External Racks, ECM, WarlordÔĽŅ

Gladiator I - Captain Brunson, Ordnance Expert, APT, DemolisherÔĽŅ

Gladiator II - Agent Kallus, Ruthless Strategist, External Racks

Squadrons (120 points) - Morna Kee, Captain Jonus, Colonel Jendon, Maarek Stele, 2x Firespray-31

 

@CptAwesomer Thanks for doing this! My total fleet points was 398. Doesn't look like you missed anything. Edit: My Vic 1 had ordnance experts

Objectives: Precision Strike, Fighter Ambush, Superior Positions

Edited by Kristjan

Share this post


Link to post
Share on other sites
7 hours ago, xero989 said:

I'm a little confused on the selection of ackbar, as you get more totally dice if you just double arch stuff, I guess he is just insurance for the times you only have a side shot?

it depends on what he flies against. Against imperials and front arc ships, he'll circle around and use Ackbar a lot. Against other types of list (like my Mothma list), he'll go for double arcs. It made his CR90s super dangerous and his peltas as well. Even when I got close to use my MC30s, he could still throw 4 reds and 2 blacks. The way he flew, all his ships were close to each other. It was impossible to have a ship survive more than 2 rounds. My own Admolando died after getting shot by the 2 peltas.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...