<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:media="http://search.yahoo.com/mrss/"><channel><title><![CDATA[Daniel Kawalsky's Code Blog]]></title><description><![CDATA[Some of my programming finds, curiosities, and projects.]]></description><link>https://blog.danielkawalsky.com/</link><image><url>https://blog.danielkawalsky.com/favicon.png</url><title>Daniel Kawalsky&apos;s Code Blog</title><link>https://blog.danielkawalsky.com/</link></image><generator>Jamify 1.0</generator><lastBuildDate>Tue, 24 Nov 2020 05:55:01 GMT</lastBuildDate><atom:link href="https://blog.danielkawalsky.com/rss/" rel="self" type="application/rss+xml"/><ttl>60</ttl><item><title><![CDATA[Mapping COVID-19 Cases by Country with React and D3]]></title><description><![CDATA[You're going to make a choropleth component like the one I used in WatchCOVID.org; that is, a choropleth map of confirmed global COVID-19 cases in React, and you'll be using SVG's to create the map and Johns Hopkins University CSSE Data Repository to populate it meaningfully. ]]></description><link>https://blog.danielkawalsky.com/mapping-covid-data/</link><guid isPermaLink="false">Ghost__Post__5fbc779a3c4f984616f88baa</guid><category><![CDATA[React]]></category><category><![CDATA[D3]]></category><category><![CDATA[COVID-19]]></category><dc:creator><![CDATA[Daniel Kawalsky]]></dc:creator><pubDate>Sun, 12 Apr 2020 03:05:00 GMT</pubDate><media:content url="https://blog.danielkawalsky.com/static/1288a4dfff086e97b5221f6b49dc3dcd/Screen-Shot-2020-11-23-at-9.11.39-PM.png" medium="image"/><content:encoded><![CDATA[<!--kg-card-begin: markdown--><img src="https://blog.danielkawalsky.com/static/1288a4dfff086e97b5221f6b49dc3dcd/Screen-Shot-2020-11-23-at-9.11.39-PM.png" alt="Mapping COVID-19 Cases by Country with React and D3"/><p>You're going to make a <a href="https://en.wikipedia.org/wiki/Choropleth_map">choropleth</a> component like the one I used in <a href="https://www.watchCOVID.org">WatchCOVID.org</a>; that is, a choropleth map of confirmed global COVID-19 cases in React, and you'll be using SVG's to create the map and <a href="https://github.com/CSSEGISandData/COVID-19">Johns Hopkins University CSSE</a> Data Repository to populate it meaningfully. D3 will help us fetch the basemap, project the basemap, and fetch the case data.</p>
<p>During setup, you'll install these:</p>
<ul>
<li><a href="https://www.npmjs.com/package/d3">D3</a>: to fetch the data</li>
<li><a href="https://www.npmjs.com/package/d3-geo">d3-geo</a>: to project the basemap coordinates into a recognizable map</li>
<li><a href="https://www.npmjs.com/package/topojson-client">topojson-client</a>: to extract coordinates from the TopoJSON basemap</li>
</ul>
<blockquote>
<p><strong>Reader</strong>: Not only the content, but the <em>format</em> of the data with which you'll be building a map in this tutorial changes rapidly. Please reach out if something doesn't work anymore so I can keep this updated.</p>
</blockquote>
<p class="fluid-image" style="flex: 1.4522144522144522 1 0"><img src="https://images2.imgbox.com/e4/3b/2JN9O1KN_o.png" alt="Mapping COVID-19 Cases by Country with React and D3" title="Final Product" maxWidth="1246"/></p>
<h2 id="setup">Setup</h2>
<ol start="0">
<li>Open 2 tabs in terminal and navigate both to the folder where you're going to create your React project.</li>
</ol>
<p><strong>Tabs 1 &#x26; 2:</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="bash"><pre class="language-bash"><code class="language-bash">$ <span class="token builtin class-name">cd</span> ~/Documents/ReactProjects
</code></pre></div>
<ol>
<li>Create a React project.</li>
</ol>
<p><strong>Tab 1:</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="bash"><pre class="language-bash"><code class="language-bash">$ npx create-react-app my-map-app
</code></pre></div>
<ol start="2">
<li>Setup both tabs.</li>
</ol>
<ul>
<li>Change the current working directory of both tabs to the project you've just created.</li>
</ul>
<p><strong>Tabs 1 &#x26; 2:</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="bash"><pre class="language-bash"><code class="language-bash">$ <span class="token builtin class-name">cd</span> my-map-app
</code></pre></div>
<ul>
<li>In one tab, start the development build of your application. In the other, install the 3 NPM packages you'll be using: <a href="https://www.npmjs.com/package/d3"><code class="language-text">D3</code></a>, <a href="https://www.npmjs.com/package/d3-geo"><code class="language-text">d3-geo</code></a>, and <a href="https://www.npmjs.com/package/topojson-client"><code class="language-text">topojson-client</code></a>.</li>
</ul>
<p><strong>Tab 1:</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="bash"><pre class="language-bash"><code class="language-bash">$ <span class="token function">npm</span> start
</code></pre></div>
<p><strong>Tab 2:</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="bash"><pre class="language-bash"><code class="language-bash">$ <span class="token function">npm</span> <span class="token function">install</span> d3 d3-geo topojson-client
</code></pre></div>
<ol start="3">
<li>Point your browser to <a href="http://localhost:3000/"><code class="language-text">http://localhost:3000/</code></a>. React should be rendering the <code class="language-text">&#x3C;App /></code> component by default.</li>
</ol>
<p class="fluid-image" style="flex: 1.8936651583710407 1 0"><img src="https://images2.imgbox.com/f2/18/hblb6E5b_o.png" alt="Mapping COVID-19 Cases by Country with React and D3" title="React default App component" maxWidth="3348"/></p>
<ol start="4">
<li>Clear out any stuff from <strong>App.js</strong> that nobody asked for.</li>
</ol>
<p>From <strong>App.js</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="diff"><pre class="language-diff"><code class="language-diff">import React from 'react';
<span class="token deleted-sign deleted"><span class="token prefix deleted">-</span><span class="token line"> import logo from './logo.svg';
</span></span>import './App.css';

function App() {
<span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> return (
</span><span class="token prefix unchanged"> </span><span class="token line">   &#x3C;div className="App">
</span></span><span class="token deleted-sign deleted"><span class="token prefix deleted">-</span><span class="token line">      &#x3C;header className="App-header">
</span><span class="token prefix deleted">-</span><span class="token line">         &#x3C;img src={logo} className="App-logo" alt="logo" />
</span><span class="token prefix deleted">-</span><span class="token line">        &#x3C;p>
</span><span class="token prefix deleted">-</span><span class="token line">          Edit &#x3C;code>src/App.js&#x3C;/code> and save to reload.
</span><span class="token prefix deleted">-</span><span class="token line">        &#x3C;/p>
</span><span class="token prefix deleted">-</span><span class="token line">        &#x3C;a
</span><span class="token prefix deleted">-</span><span class="token line">          className="App-link"
</span><span class="token prefix deleted">-</span><span class="token line">          href="https://reactjs.org"
</span><span class="token prefix deleted">-</span><span class="token line">          target="_blank"
</span><span class="token prefix deleted">-</span><span class="token line">          rel="noopener noreferrer"
</span><span class="token prefix deleted">-</span><span class="token line">        >
</span><span class="token prefix deleted">-</span><span class="token line">          Learn React
</span><span class="token prefix deleted">-</span><span class="token line">        &#x3C;/a>
</span><span class="token prefix deleted">-</span><span class="token line">      &#x3C;/header>
</span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">   &#x3C;/div>
</span><span class="token prefix unchanged"> </span><span class="token line"> );
</span></span>}

export default App;
</code></pre></div>
<p>Remove all the rules from <strong>App.css</strong> so you're left with just the class <code class="language-text">App</code>.</p>
<p><strong>App.css</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="css"><pre class="language-css"><code class="language-css"><span class="token selector">.App</span> <span class="token punctuation">{</span>
  
<span class="token punctuation">}</span>
</code></pre></div>
<ol start="4">
<li>Create <strong>src/Map.jsx</strong>. Here's where you'll do your dirty work.</li>
</ol>
<p>Your file structure should have these bones.</p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="tree"><pre class="language-tree"><code class="language-tree">.
├── README.md
├── package-lock.json
├── package.json
├── public
└── src
    ├── App.css
    ├── App.js
    ├── Map.jsx
    ├── index.css
    └── index.js
</code></pre></div>
<ol start="5">
<li>Import <strong>Map.jsx</strong> into <strong>App.js</strong></li>
</ol>
<p><strong>App.js</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="diff"><pre class="language-diff"><code class="language-diff"><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> import React from 'react';
</span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line"> import Map from './Map';
</span></span>
<span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> import './App.css';
</span></span>
<span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> function App() {
</span><span class="token prefix unchanged"> </span><span class="token line">   return (
</span><span class="token prefix unchanged"> </span><span class="token line">     &#x3C;div className="App">
</span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line">       &#x3C;Map />
</span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">     &#x3C;/div>
</span><span class="token prefix unchanged"> </span><span class="token line">   );
</span><span class="token prefix unchanged"> </span><span class="token line"> }
</span></span>
<span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> export default App;
</span></span></code></pre></div>
<ol start="6">
<li>Define <strong>Map.jsx</strong></li>
</ol>
<p><strong>Map.jsx</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">import</span> React <span class="token keyword">from</span> <span class="token string">'react'</span><span class="token punctuation">;</span>

<span class="token keyword">export</span> <span class="token keyword">default</span> <span class="token keyword">class</span> <span class="token class-name">Map</span> <span class="token keyword">extends</span> <span class="token class-name">React<span class="token punctuation">.</span>Component</span> <span class="token punctuation">{</span>
  <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token parameter">props</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
    <span class="token keyword">super</span><span class="token punctuation">(</span>props<span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token keyword">this</span><span class="token punctuation">.</span>state <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span>
  <span class="token punctuation">}</span>

  <span class="token function">componentDidMount</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
  <span class="token punctuation">}</span>

  <span class="token function">render</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
    <span class="token keyword">return</span> <span class="token punctuation">(</span>
      <span class="token tag"><span class="token tag"><span class="token punctuation">&#x3C;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>Map<span class="token punctuation">"</span></span> <span class="token punctuation">></span></span><span class="token plain-text">
      	MAPONENT
      </span><span class="token tag"><span class="token tag"><span class="token punctuation">&#x3C;/</span>div</span><span class="token punctuation">></span></span>
    <span class="token punctuation">)</span><span class="token punctuation">;</span>
  <span class="token punctuation">}</span>
<span class="token punctuation">}</span>

</code></pre></div>
<h2 id="makeamap">Make a Map</h2>
<p>You're going to import an open-source SVG basemap of the world's countries' borders. You'll fill these borders in with gradients of a color to present the distribution of COVID-19 cases. You'll use this <a href="https://github.com/topojson/world-atlas#countries-110m.json">110 meter-quality map</a> from the TopoJSON <a href="https://github.com/topojson/world-atlas">World Atlas repository</a>. Here's the download URL:</p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="text"><pre class="language-text"><code class="language-text">https://cdn.jsdelivr.net/npm/world-atlas@2/countries-110m.json
</code></pre></div>
<h3 id="grabthebasemaptopojson">Grab the Basemap TopoJSON</h3>
<p>Import the TopoJSON into your Map component using D3, then log its contents to the console.</p>
<p><strong>Map.jsx</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="diff"><pre class="language-diff"><code class="language-diff"><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> import React from 'react';
</span></span>
<span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line"> import * as d3 from 'd3';
</span></span>
<span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> export default class Map extends React.Component {
</span></span>
<span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">   ...
</span></span>
<span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line">   componentDidMount() {
</span><span class="token prefix inserted">+</span><span class="token line">     d3.json("https://cdn.jsdelivr.net/npm/world-atlas@2/countries-110m.json")
</span><span class="token prefix inserted">+</span><span class="token line">       .then(json => {
</span><span class="token prefix inserted">+</span><span class="token line">         console.log(json);
</span><span class="token prefix inserted">+</span><span class="token line">       });
</span><span class="token prefix inserted">+</span><span class="token line">   }
</span></span>
<span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">   ...
</span><span class="token prefix unchanged"> </span><span class="token line"> }
</span></span></code></pre></div>
<p>Open your browser console to and expand the object you've just logged.</p>
<p>Expand (<strong>Object</strong>) > <code class="language-text">objects</code> > <code class="language-text">countries</code> > <code class="language-text">geometries</code>:</p>
<p class="fluid-image" style="flex: 2.2773722627737225 1 0"><img src="https://i.ibb.co/rcPvmfX/topojson-level-2.png" alt="Mapping COVID-19 Cases by Country with React and D3" title="TopoJSON log ouput" maxWidth="1248"/></p>
<p>Notice the array under <code class="language-text">geometries</code> has length 177. Those are distinct country entries, and because of your basemap is of the lowest resolution, you overlook some exceptionally small countries in this tutorial.</p>
<p>Now expand some of the <code class="language-text">geometries</code> elements to find the country name associated with each entry. Try (<strong>Object</strong>) > <code class="language-text">objects</code> > <code class="language-text">countries</code> > <code class="language-text">geometries</code> > <code class="language-text">0</code> > <code class="language-text">properties</code>, where you'll find the value <code class="language-text">name</code>. You can log the 0th element's name by modifying the <code class="language-text">componentDidMount()</code> callback (above) to instead log:</p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx">console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>json<span class="token punctuation">.</span>objects<span class="token punctuation">.</span>countries<span class="token punctuation">.</span>geometries<span class="token punctuation">[</span> <span class="token number">0</span> <span class="token punctuation">]</span><span class="token punctuation">.</span>properties<span class="token punctuation">.</span>name<span class="token punctuation">)</span><span class="token punctuation">;</span>
</code></pre></div>
<p class="fluid-image" style="flex: 1.7002724795640327 1 0"><img src="https://images2.imgbox.com/d4/a4/tlBgBkC5_o.png" alt="Mapping COVID-19 Cases by Country with React and D3" title="TopoJSON log ouput to name" maxWidth="1248"/></p>
<h3 id="convertthetopojsontocoordinates">Convert the TopoJSON to Coordinates</h3>
<p>If you look at the <code class="language-text">arcs</code> field, you'll find an array of <a href="https://en.wikipedia.org/wiki/Delta_encoding">delta-encoded</a> instructions on how to draw that particular polygon. The <code class="language-text">topojson-client</code> package you downloaded turns the TopoJSON into a map you can recognize by converting the arcs to SVG paths.</p>
<p>If you're curious about how <code class="language-text">topojson-client</code> helps, notice how the <code class="language-text">feature()</code> function you'll import from <code class="language-text">topojson-client</code> extracts the <em>coordinates</em> from the delta-encoded arcs by logging the following:</p>
<p><strong>Map.jsx</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="diff"><pre class="language-diff"><code class="language-diff"><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line"> import { feature } from "topojson-client"
</span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">  ...
</span><span class="token prefix unchanged"> </span><span class="token line">   componentDidMount() {
</span><span class="token prefix unchanged"> </span><span class="token line">     ...
</span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line">         console.log(feature(json, json.objects.countries));
</span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">     ...
</span><span class="token prefix unchanged"> </span><span class="token line">   }
</span></span></code></pre></div>
<p class="fluid-image" style="flex: 0.7907348242811502 1 0"><img src="https://images2.imgbox.com/58/63/KRDrU2Tw_o.png" alt="Mapping COVID-19 Cases by Country with React and D3" title="TopoJSON feature-ized" maxWidth="990"/></p>
<h3 id="projectthecoordinatesandvisualizethesvg">Project the Coordinates and Visualize the SVG</h3>
<p>The changes you make below will connect the coordinates to draw each country with a series of SVGs.</p>
<p>Make the following changes to <strong>Map.jsx</strong>:</p>
<p><strong>Map.jsx</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">import</span> React <span class="token keyword">from</span> <span class="token string">'react'</span><span class="token punctuation">;</span>

<span class="token keyword">import</span> <span class="token operator">*</span> <span class="token keyword">as</span> d3 <span class="token keyword">from</span> <span class="token string">'d3'</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token punctuation">{</span> geoMercator<span class="token punctuation">,</span> geoPath <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">"d3-geo"</span>
<span class="token keyword">import</span> <span class="token punctuation">{</span> feature <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">"topojson-client"</span>

<span class="token comment">// 6</span>
<span class="token keyword">const</span> projection <span class="token operator">=</span> <span class="token function">geoMercator</span><span class="token punctuation">(</span><span class="token punctuation">)</span>
  <span class="token punctuation">.</span><span class="token function">scale</span><span class="token punctuation">(</span><span class="token number">100</span><span class="token punctuation">)</span>
  <span class="token punctuation">.</span><span class="token function">translate</span><span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token number">400</span><span class="token punctuation">,</span> <span class="token number">300</span><span class="token punctuation">]</span><span class="token punctuation">)</span>

<span class="token keyword">export</span> <span class="token keyword">default</span> <span class="token keyword">class</span> <span class="token class-name">Map</span> <span class="token keyword">extends</span> <span class="token class-name">React<span class="token punctuation">.</span>Component</span> <span class="token punctuation">{</span>
  <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token parameter">props</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
    <span class="token keyword">super</span><span class="token punctuation">(</span>props<span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token keyword">this</span><span class="token punctuation">.</span>state <span class="token operator">=</span> <span class="token punctuation">{</span>
      <span class="token comment">// 1</span>
      countryShapes<span class="token operator">:</span> <span class="token keyword">null</span><span class="token punctuation">,</span>
    <span class="token punctuation">}</span><span class="token punctuation">;</span>
  <span class="token punctuation">}</span>

  <span class="token function">componentDidMount</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
    d3<span class="token punctuation">.</span><span class="token function">json</span><span class="token punctuation">(</span><span class="token string">"https://cdn.jsdelivr.net/npm/world-atlas@2/countries-110m.json"</span><span class="token punctuation">)</span>
      <span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span><span class="token parameter">json</span> <span class="token operator">=></span> <span class="token punctuation">{</span>
        <span class="token comment">// 4</span>
        <span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">setState</span><span class="token punctuation">(</span><span class="token punctuation">{</span>
          countryShapes<span class="token operator">:</span> <span class="token function">feature</span><span class="token punctuation">(</span>json<span class="token punctuation">,</span> json<span class="token punctuation">.</span>objects<span class="token punctuation">.</span>countries<span class="token punctuation">)</span><span class="token punctuation">.</span>features<span class="token punctuation">,</span>
        <span class="token punctuation">}</span><span class="token punctuation">)</span>
      <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
  <span class="token punctuation">}</span>

  <span class="token function">render</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
    <span class="token keyword">return</span> <span class="token punctuation">(</span>
      <span class="token tag"><span class="token tag"><span class="token punctuation">&#x3C;</span>div</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>Map<span class="token punctuation">"</span></span> <span class="token punctuation">></span></span><span class="token plain-text">
        </span><span class="token punctuation">{</span><span class="token comment">/* 2 */</span><span class="token punctuation">}</span><span class="token plain-text">
        </span><span class="token tag"><span class="token tag"><span class="token punctuation">&#x3C;</span>svg</span> <span class="token attr-name">width</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token number">1024</span><span class="token punctuation">}</span></span> <span class="token attr-name">height</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token number">650</span><span class="token punctuation">}</span></span> <span class="token attr-name">viewBox</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>0 0 1024 650<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token plain-text">
          </span><span class="token tag"><span class="token tag"><span class="token punctuation">&#x3C;</span>g</span> <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>countries-group<span class="token punctuation">"</span></span><span class="token punctuation">></span></span><span class="token plain-text">
            </span><span class="token punctuation">{</span><span class="token comment">// 3</span>
              <span class="token keyword">this</span><span class="token punctuation">.</span>state<span class="token punctuation">.</span>countryShapes <span class="token operator">?</span>
                <span class="token comment">// 5</span>
                <span class="token keyword">this</span><span class="token punctuation">.</span>state<span class="token punctuation">.</span>countryShapes<span class="token punctuation">.</span><span class="token function">map</span><span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token parameter">featureElement<span class="token punctuation">,</span> index</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">(</span>
                  <span class="token operator">&#x3C;</span>path
                    key<span class="token operator">=</span><span class="token punctuation">{</span><span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">country-svg-</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>index<span class="token interpolation-punctuation punctuation">}</span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">}</span>
                    <span class="token comment">// 7</span>
                    d<span class="token operator">=</span><span class="token punctuation">{</span><span class="token function">geoPath</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">projection</span><span class="token punctuation">(</span>projection<span class="token punctuation">)</span><span class="token punctuation">(</span>featureElement<span class="token punctuation">)</span><span class="token punctuation">}</span>
                    className<span class="token operator">=</span><span class="token string">"country"</span>
                    <span class="token comment">// 8</span>
                    fill<span class="token operator">=</span><span class="token punctuation">{</span><span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">rgba(255,0,255,0.5)</span><span class="token template-punctuation string">`</span></span><span class="token punctuation">}</span>
                    stroke<span class="token operator">=</span><span class="token string">"black"</span>
                    strokeWidth<span class="token operator">=</span><span class="token punctuation">{</span><span class="token number">0.25</span><span class="token punctuation">}</span>
                  <span class="token operator">/</span><span class="token operator">></span>
                <span class="token punctuation">)</span><span class="token punctuation">)</span> <span class="token operator">:</span> <span class="token keyword">null</span>
            <span class="token punctuation">}</span><span class="token plain-text">
          </span><span class="token tag"><span class="token tag"><span class="token punctuation">&#x3C;/</span>g</span><span class="token punctuation">></span></span><span class="token plain-text">
        </span><span class="token tag"><span class="token tag"><span class="token punctuation">&#x3C;/</span>svg</span><span class="token punctuation">></span></span><span class="token plain-text">
      </span><span class="token tag"><span class="token tag"><span class="token punctuation">&#x3C;/</span>div</span><span class="token punctuation">></span></span>
    <span class="token punctuation">)</span><span class="token punctuation">;</span>
  <span class="token punctuation">}</span>
<span class="token punctuation">}</span>
</code></pre></div>
<ol>
<li>Define the object <code class="language-text">countryShapes</code> variable in your component state with default value <code class="language-text">null</code>. This will be re-initialized in <code class="language-text">componentDidMount()</code>.</li>
<li>By default, render an SVG image with a <code class="language-text">&#x3C;g /></code> element called <code class="language-text">countries-group</code>.</li>
<li>Until the <code class="language-text">countryShapes</code> state variable is defined, don't render any <code class="language-text">&#x3C;path /></code>s within the <code class="language-text">&#x3C;g /></code> element.</li>
<li>After the <code class="language-text">Map</code> component's first render, set the <code class="language-text">countryShapes</code> state variable to the features of the TopoJSON you fetched and inspected just above this code block.</li>
<li>The <code class="language-text">countryShapes</code> state variable becomes truthy, turning each <code class="language-text">featureElement</code>, which contains the coordiate borders of each country, into an array of <code class="language-text">&#x3C;path /></code> components.</li>
<li>Use the Mercator projection provided by <code class="language-text">d3-geo</code>, scale the image to 100% (instead of the default 150%) and translate the map right on the x-axis by 400 down the y-axis by 300.</li>
<li>The customized Mercator <code class="language-text">projection</code>, provided to <code class="language-text">geoPath().projection()</code>, returns a function that takes a <code class="language-text">featureElement</code>. This function returns <code class="language-text">featureElement</code>'s projected coordinates. Give this to <code class="language-text">&#x3C;path /></code> to draw a country's borders.</li>
<li>Uniformly fill each <code class="language-text">&#x3C;path /></code> polygon with a pink color defined by <code class="language-text">rgba(255, 0, 255, 0.5)</code>.</li>
</ol>
<h3 id="makethemapintoachoroplethmap">Make the Map into a Choropleth Map</h3>
<p>To make this a choropleth map, you will show the varying quantities of cases by country by varying base color's opacity. Let's make a helper function that uses the name of the country you pull from the TopoJSON (and sample COVID-19 case data) to produce a color.</p>
<p>Define the sample data below <code class="language-text">projection</code> in <strong>Map.jsx</strong>:</p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">const</span> sampleData <span class="token operator">=</span> <span class="token punctuation">{</span>
  <span class="token string">"Fiji"</span><span class="token operator">:</span> <span class="token number">3074</span><span class="token punctuation">,</span>
  <span class="token string">"Tanzania"</span><span class="token operator">:</span> <span class="token number">9138</span><span class="token punctuation">,</span>
  <span class="token string">"W. Sahara"</span><span class="token operator">:</span> <span class="token number">4143</span><span class="token punctuation">,</span>
  <span class="token string">"Canada"</span><span class="token operator">:</span> <span class="token number">9726</span><span class="token punctuation">,</span>
  <span class="token string">"United States of America"</span><span class="token operator">:</span> <span class="token number">2439</span><span class="token punctuation">,</span>
  <span class="token string">"Kazakhstan"</span><span class="token operator">:</span> <span class="token number">5090</span><span class="token punctuation">,</span>
  <span class="token string">"Uzbekistan"</span><span class="token operator">:</span> <span class="token number">7353</span><span class="token punctuation">,</span>
  <span class="token string">"Papua New Guinea"</span><span class="token operator">:</span> <span class="token number">6460</span><span class="token punctuation">,</span>
  <span class="token string">"Indonesia"</span><span class="token operator">:</span> <span class="token number">6747</span><span class="token punctuation">,</span>
  <span class="token string">"Argentina"</span><span class="token operator">:</span> <span class="token number">9</span><span class="token punctuation">,</span>
  <span class="token string">"Chile"</span><span class="token operator">:</span> <span class="token number">5864</span><span class="token punctuation">,</span>
  <span class="token string">"Dem. Rep. Congo"</span><span class="token operator">:</span> <span class="token number">6845</span><span class="token punctuation">,</span>
  <span class="token string">"Somalia"</span><span class="token operator">:</span> <span class="token number">192</span><span class="token punctuation">,</span>
  <span class="token string">"Kenya"</span><span class="token operator">:</span> <span class="token number">6651</span><span class="token punctuation">,</span>
  <span class="token string">"Sudan"</span><span class="token operator">:</span> <span class="token number">346</span><span class="token punctuation">,</span>
  <span class="token string">"Chad"</span><span class="token operator">:</span> <span class="token number">6251</span><span class="token punctuation">,</span>
  <span class="token string">"Haiti"</span><span class="token operator">:</span> <span class="token number">1143</span><span class="token punctuation">,</span>
  <span class="token string">"Dominican Rep."</span><span class="token operator">:</span> <span class="token number">2436</span><span class="token punctuation">,</span>
  <span class="token string">"Russia"</span><span class="token operator">:</span> <span class="token number">4703</span><span class="token punctuation">,</span>
  <span class="token string">"Bahamas"</span><span class="token operator">:</span> <span class="token number">5904</span><span class="token punctuation">,</span>
  <span class="token string">"Falkland Is."</span><span class="token operator">:</span> <span class="token number">3269</span><span class="token punctuation">,</span>
  <span class="token string">"Norway"</span><span class="token operator">:</span> <span class="token number">8617</span><span class="token punctuation">,</span>
  <span class="token string">"Greenland"</span><span class="token operator">:</span> <span class="token number">3984</span><span class="token punctuation">,</span>
  <span class="token string">"Fr. S. Antarctic Lands"</span><span class="token operator">:</span> <span class="token number">8599</span><span class="token punctuation">,</span>
  <span class="token string">"Timor-Leste"</span><span class="token operator">:</span> <span class="token number">6780</span><span class="token punctuation">,</span>
  <span class="token string">"South Africa"</span><span class="token operator">:</span> <span class="token number">4176</span><span class="token punctuation">,</span>
  <span class="token string">"Lesotho"</span><span class="token operator">:</span> <span class="token number">1642</span><span class="token punctuation">,</span>
  <span class="token string">"Mexico"</span><span class="token operator">:</span> <span class="token number">4020</span><span class="token punctuation">,</span>
  <span class="token string">"Uruguay"</span><span class="token operator">:</span> <span class="token number">8650</span><span class="token punctuation">,</span>
  <span class="token string">"Brazil"</span><span class="token operator">:</span> <span class="token number">2984</span><span class="token punctuation">,</span>
  <span class="token string">"Bolivia"</span><span class="token operator">:</span> <span class="token number">1736</span><span class="token punctuation">,</span>
  <span class="token string">"Peru"</span><span class="token operator">:</span> <span class="token number">2555</span><span class="token punctuation">,</span>
  <span class="token string">"Colombia"</span><span class="token operator">:</span> <span class="token number">4452</span><span class="token punctuation">,</span>
  <span class="token string">"Panama"</span><span class="token operator">:</span> <span class="token number">4131</span><span class="token punctuation">,</span>
  <span class="token string">"Costa Rica"</span><span class="token operator">:</span> <span class="token number">91</span><span class="token punctuation">,</span>
  <span class="token string">"Nicaragua"</span><span class="token operator">:</span> <span class="token number">7043</span><span class="token punctuation">,</span>
  <span class="token string">"Honduras"</span><span class="token operator">:</span> <span class="token number">6387</span><span class="token punctuation">,</span>
  <span class="token string">"El Salvador"</span><span class="token operator">:</span> <span class="token number">1059</span><span class="token punctuation">,</span>
  <span class="token string">"Guatemala"</span><span class="token operator">:</span> <span class="token number">3389</span><span class="token punctuation">,</span>
  <span class="token string">"Belize"</span><span class="token operator">:</span> <span class="token number">1865</span><span class="token punctuation">,</span>
  <span class="token string">"Venezuela"</span><span class="token operator">:</span> <span class="token number">7918</span><span class="token punctuation">,</span>
  <span class="token string">"Guyana"</span><span class="token operator">:</span> <span class="token number">9731</span><span class="token punctuation">,</span>
  <span class="token string">"Suriname"</span><span class="token operator">:</span> <span class="token number">7663</span><span class="token punctuation">,</span>
  <span class="token string">"France"</span><span class="token operator">:</span> <span class="token number">8582</span><span class="token punctuation">,</span>
  <span class="token string">"Ecuador"</span><span class="token operator">:</span> <span class="token number">1889</span><span class="token punctuation">,</span>
  <span class="token string">"Puerto Rico"</span><span class="token operator">:</span> <span class="token number">6067</span><span class="token punctuation">,</span>
  <span class="token string">"Jamaica"</span><span class="token operator">:</span> <span class="token number">5362</span><span class="token punctuation">,</span>
  <span class="token string">"Cuba"</span><span class="token operator">:</span> <span class="token number">6440</span><span class="token punctuation">,</span>
  <span class="token string">"Zimbabwe"</span><span class="token operator">:</span> <span class="token number">1816</span><span class="token punctuation">,</span>
  <span class="token string">"Botswana"</span><span class="token operator">:</span> <span class="token number">5960</span><span class="token punctuation">,</span>
  <span class="token string">"Namibia"</span><span class="token operator">:</span> <span class="token number">9102</span><span class="token punctuation">,</span>
  <span class="token string">"Senegal"</span><span class="token operator">:</span> <span class="token number">9869</span><span class="token punctuation">,</span>
  <span class="token string">"Mali"</span><span class="token operator">:</span> <span class="token number">2561</span><span class="token punctuation">,</span>
  <span class="token string">"Mauritania"</span><span class="token operator">:</span> <span class="token number">9969</span><span class="token punctuation">,</span>
  <span class="token string">"Benin"</span><span class="token operator">:</span> <span class="token number">6901</span><span class="token punctuation">,</span>
  <span class="token string">"Niger"</span><span class="token operator">:</span> <span class="token number">5985</span><span class="token punctuation">,</span>
  <span class="token string">"Nigeria"</span><span class="token operator">:</span> <span class="token number">4014</span><span class="token punctuation">,</span>
  <span class="token string">"Cameroon"</span><span class="token operator">:</span> <span class="token number">3428</span><span class="token punctuation">,</span>
  <span class="token string">"Togo"</span><span class="token operator">:</span> <span class="token number">4304</span><span class="token punctuation">,</span>
  <span class="token string">"Ghana"</span><span class="token operator">:</span> <span class="token number">8478</span><span class="token punctuation">,</span>
  <span class="token string">"Côte d'Ivoire"</span><span class="token operator">:</span> <span class="token number">1051</span><span class="token punctuation">,</span>
  <span class="token string">"Guinea"</span><span class="token operator">:</span> <span class="token number">5548</span><span class="token punctuation">,</span>
  <span class="token string">"Guinea-Bissau"</span><span class="token operator">:</span> <span class="token number">7992</span><span class="token punctuation">,</span>
  <span class="token string">"Liberia"</span><span class="token operator">:</span> <span class="token number">6675</span><span class="token punctuation">,</span>
  <span class="token string">"Sierra Leone"</span><span class="token operator">:</span> <span class="token number">4517</span><span class="token punctuation">,</span>
  <span class="token string">"Burkina Faso"</span><span class="token operator">:</span> <span class="token number">4694</span><span class="token punctuation">,</span>
  <span class="token string">"Central African Rep."</span><span class="token operator">:</span> <span class="token number">1429</span><span class="token punctuation">,</span>
  <span class="token string">"Congo"</span><span class="token operator">:</span> <span class="token number">7581</span><span class="token punctuation">,</span>
  <span class="token string">"Gabon"</span><span class="token operator">:</span> <span class="token number">449</span><span class="token punctuation">,</span>
  <span class="token string">"Eq. Guinea"</span><span class="token operator">:</span> <span class="token number">4241</span><span class="token punctuation">,</span>
  <span class="token string">"Zambia"</span><span class="token operator">:</span> <span class="token number">3389</span><span class="token punctuation">,</span>
  <span class="token string">"Malawi"</span><span class="token operator">:</span> <span class="token number">5880</span><span class="token punctuation">,</span>
<span class="token punctuation">}</span><span class="token punctuation">;</span>
</code></pre></div>
<p>Define the helper function <code class="language-text">getCountryFill()</code> below <code class="language-text">sampleData</code> in <strong>Map.jsx</strong>:</p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">function</span> <span class="token function">getCountryFill</span><span class="token punctuation">(</span><span class="token parameter">featureElement</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
  <span class="token keyword">let</span> name <span class="token operator">=</span> featureElement<span class="token punctuation">.</span>properties<span class="token punctuation">.</span>name<span class="token punctuation">;</span>
  <span class="token keyword">let</span> value <span class="token operator">=</span> sampleData<span class="token punctuation">[</span>name<span class="token punctuation">]</span><span class="token punctuation">;</span>
  <span class="token keyword">if</span> <span class="token punctuation">(</span>value<span class="token punctuation">)</span> <span class="token punctuation">{</span>
    <span class="token keyword">let</span> opacity <span class="token operator">=</span> value <span class="token operator">/</span> <span class="token number">10000</span><span class="token punctuation">;</span>
    <span class="token keyword">return</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">rgba(255, 0, 255, </span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>opacity<span class="token interpolation-punctuation punctuation">}</span></span><span class="token string">)</span><span class="token template-punctuation string">`</span></span><span class="token punctuation">;</span> <span class="token comment">// 2</span>
  <span class="token punctuation">}</span>
  <span class="token keyword">return</span> <span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">rgba(200, 200, 200, 1)</span><span class="token template-punctuation string">`</span></span><span class="token punctuation">;</span> <span class="token comment">// 1</span>
<span class="token punctuation">}</span>
</code></pre></div>
<ol>
<li>If there is no value in the data, use a default gray value to indicate "no data."</li>
<li>If the value is present in the array, use your pink color from before and vary the opacity.</li>
</ol>
<p>The values in the sample data are randomly positive integers less than 10,000. For now, the opacity is arbitrarily defined by "<code class="language-text">value / 10000</code>." You'll make this function less trivial further on.</p>
<p>Let's make this choropleth map happen. For your <code class="language-text">&#x3C;path /></code>, redefine the <code class="language-text">fill</code> parameter:</p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token operator">...</span>
	<span class="token tag"><span class="token tag"><span class="token punctuation">&#x3C;</span>path</span>
      <span class="token attr-name">key</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token template-string"><span class="token template-punctuation string">`</span><span class="token string">country-svg-</span><span class="token interpolation"><span class="token interpolation-punctuation punctuation">${</span>index<span class="token interpolation-punctuation punctuation">}</span></span><span class="token template-punctuation string">`</span></span><span class="token punctuation">}</span></span>
      <span class="token attr-name">d</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token function">geoPath</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">projection</span><span class="token punctuation">(</span>projection<span class="token punctuation">)</span><span class="token punctuation">(</span>featureElement<span class="token punctuation">)</span><span class="token punctuation">}</span></span>
      <span class="token attr-name">className</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>country<span class="token punctuation">"</span></span>
      <span class="token attr-name">fill</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token function">getCountryFill</span><span class="token punctuation">(</span>featureElement<span class="token punctuation">)</span><span class="token punctuation">}</span></span>
      <span class="token attr-name">stroke</span><span class="token attr-value"><span class="token punctuation attr-equals">=</span><span class="token punctuation">"</span>black<span class="token punctuation">"</span></span>
      <span class="token attr-name">strokeWidth</span><span class="token script language-javascript"><span class="token script-punctuation punctuation">=</span><span class="token punctuation">{</span><span class="token number">0.25</span><span class="token punctuation">}</span></span>
    <span class="token punctuation">/></span></span>
<span class="token operator">...</span>
</code></pre></div>
<p class="fluid-image" style="flex: 1.5966666666666667 1 0"><img src="https://images2.imgbox.com/15/c7/9ZxHBi8w_o.png" alt="Mapping COVID-19 Cases by Country with React and D3" title="Basic Choropleth Map with Mercator Projection using Sample Data" maxWidth="1916"/></p>
<h2 id="useupdateddata">Use Updated Data</h2>
<p>You're going to present current COVID-19 case data, so I'm not going to show any more screenshots of what you should expect your map's colors to look like. That said, let's move onto real data! Open <a href="https://github.com/CSSEGISandData/COVID-19/blob/master/csse_COVID_19_data/csse_COVID_19_time_series/time_series_COVID19_confirmed_global.csv">Johns Hopkins' updated time series-confirmed case data</a> in another tab.</p>
<h3 id="understandthedata">Understand the Data</h3>
<p>It may show up as a table, or it could show up raw, <a href="https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_confirmed_global.csv">like this</a>. Anyhow, take a look at the columns. See that they start with [Province/State, Country/Region, Lat, Long] and then continue with dates? This is your schema. With each day, you will (predictably) have one more column.</p>
<table>
<thead>
<tr>
<th>Province/State</th>
<th>Country/Region</th>
<th>Lat</th>
<th>Long</th>
<th>...</th>
<th>1/22/20</th>
<th>1/23/20</th>
<th>...</th>
</tr>
</thead>
<tbody/>
</table>
<p>Now look at the rows. See that many start with a <code class="language-text">,</code> (or a blank cell, if you're looking at the pretty version)? That suggests that you don't have "<code class="language-text">Province/State</code>" breakdowns (first column) for every "<code class="language-text">Country/Region</code>" (second column). Because the map is only specific enough to show countries, you'll reduce each countries' province's daily values into one value for the whole country; for example, Australian Capital Territory, New South Wales, Northern Territory, Queensland, South Australia, Tasmania, Victoria, and Western Australia will all count towards one total for Australia.</p>
<h4 id="arg">ARG...</h4>
<p>Before you go ahead and start reducing, I need to point out something frustrating about this data. The country names here are different from the country names in your TopoJSON! I'll present a workaround later on, and you can take it from there. Let's grab the data.</p>
<h3 id="grabthedata">Grab the Data</h3>
<p>In <strong>Map.jsx</strong>, download the raw CSV data using <code class="language-text">d3.csv()</code> and log it to the console:</p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token function">componentDidMount</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
    d3<span class="token punctuation">.</span><span class="token function">json</span><span class="token punctuation">(</span><span class="token string">"https://cdn.jsdelivr.net/npm/world-atlas@2/countries-110m.json"</span><span class="token punctuation">)</span>
      <span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span><span class="token parameter">json</span> <span class="token operator">=></span> <span class="token punctuation">{</span>
        <span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">setState</span><span class="token punctuation">(</span><span class="token punctuation">{</span>
          countryShapes<span class="token operator">:</span> <span class="token function">feature</span><span class="token punctuation">(</span>json<span class="token punctuation">,</span> json<span class="token punctuation">.</span>objects<span class="token punctuation">.</span>countries<span class="token punctuation">)</span><span class="token punctuation">.</span>features<span class="token punctuation">,</span>
        <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    d3<span class="token punctuation">.</span><span class="token function">csv</span><span class="token punctuation">(</span><span class="token string">"https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_confirmed_global.csv"</span><span class="token punctuation">)</span>
      <span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span><span class="token parameter">csvData</span> <span class="token operator">=></span> <span class="token punctuation">{</span>
        console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>csvData<span class="token punctuation">)</span><span class="token punctuation">;</span>
        console<span class="token punctuation">.</span><span class="token function">log</span><span class="token punctuation">(</span>csvData<span class="token punctuation">[</span><span class="token number">0</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
      <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
  <span class="token punctuation">}</span><span class="token punctuation">)</span>
<span class="token punctuation">}</span>
</code></pre></div>
<p class="fluid-image" style="flex: 0.9943582510578279 1 0"><img src="https://images2.imgbox.com/f4/6e/l53BNT1o_o.jpg" alt="Mapping COVID-19 Cases by Country with React and D3" title="CSV Data" maxWidth="1410"/></p>
<p>Nice! It gives us the names of the columns. All in all, the CSV data you're dealing with:</p>
<ul>
<li>is an array of objects wherein each key is a column name.</li>
<li>has a field <code class="language-text">columns</code>, which is itself an array of columns names.</li>
</ul>
<p>Moving on...</p>
<h3 id="consolidatethedata">Consolidate the Data</h3>
<p>Using the country name in each array element, reduce the last day's confirmed case count across each country's. Your <strong>Map.jsx</strong> should look something like this:</p>
<p><strong>Map.jsx</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="jsx"><pre class="language-jsx"><code class="language-jsx"><span class="token keyword">import</span> React <span class="token keyword">from</span> <span class="token string">'react'</span><span class="token punctuation">;</span>

<span class="token keyword">import</span> <span class="token operator">*</span> <span class="token keyword">as</span> d3 <span class="token keyword">from</span> <span class="token string">'d3'</span><span class="token punctuation">;</span>
<span class="token keyword">import</span> <span class="token punctuation">{</span> geoMercator<span class="token punctuation">,</span> geoPath <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">"d3-geo"</span>
<span class="token keyword">import</span> <span class="token punctuation">{</span> feature <span class="token punctuation">}</span> <span class="token keyword">from</span> <span class="token string">"topojson-client"</span>

<span class="token keyword">const</span> projection <span class="token operator">=</span> <span class="token function">geoMercator</span><span class="token punctuation">(</span><span class="token punctuation">)</span>
  <span class="token punctuation">.</span><span class="token function">scale</span><span class="token punctuation">(</span><span class="token number">100</span><span class="token punctuation">)</span>
  <span class="token punctuation">.</span><span class="token function">translate</span><span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token number">400</span><span class="token punctuation">,</span> <span class="token number">300</span><span class="token punctuation">]</span><span class="token punctuation">)</span>

<span class="token keyword">const</span> sampleData <span class="token operator">=</span> <span class="token punctuation">{</span>
  <span class="token operator">...</span>
<span class="token punctuation">}</span><span class="token punctuation">;</span>

<span class="token keyword">function</span> <span class="token function">getCountryFill</span><span class="token punctuation">(</span><span class="token parameter">featureElement</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
  <span class="token operator">...</span>
<span class="token punctuation">}</span>

<span class="token keyword">export</span> <span class="token keyword">default</span> <span class="token keyword">class</span> <span class="token class-name">Map</span> <span class="token keyword">extends</span> <span class="token class-name">React<span class="token punctuation">.</span>Component</span> <span class="token punctuation">{</span>
  <span class="token function">constructor</span><span class="token punctuation">(</span><span class="token parameter">props</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
    <span class="token keyword">super</span><span class="token punctuation">(</span>props<span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token keyword">this</span><span class="token punctuation">.</span>state <span class="token operator">=</span> <span class="token punctuation">{</span>
      countryShapes<span class="token operator">:</span> <span class="token keyword">null</span><span class="token punctuation">,</span>
      countryDataset<span class="token operator">:</span> <span class="token keyword">null</span><span class="token punctuation">,</span>
    <span class="token punctuation">}</span><span class="token punctuation">;</span>
  <span class="token punctuation">}</span>

  <span class="token comment">// 1</span>
  <span class="token keyword">async</span> <span class="token function">componentDidMount</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
    <span class="token keyword">var</span> countryShapes <span class="token operator">=</span> <span class="token keyword">null</span><span class="token punctuation">;</span>
    <span class="token keyword">var</span> countryDataset <span class="token operator">=</span> <span class="token keyword">null</span><span class="token punctuation">;</span>
    <span class="token comment">// 2</span>
    <span class="token keyword">await</span> d3<span class="token punctuation">.</span><span class="token function">json</span><span class="token punctuation">(</span><span class="token string">"https://cdn.jsdelivr.net/npm/world-atlas@2/countries-110m.json"</span><span class="token punctuation">)</span>
      <span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span><span class="token keyword">async</span> <span class="token punctuation">(</span><span class="token parameter">json</span><span class="token punctuation">)</span> <span class="token operator">=></span> <span class="token punctuation">{</span>
        countryShapes <span class="token operator">=</span> <span class="token function">feature</span><span class="token punctuation">(</span>json<span class="token punctuation">,</span> json<span class="token punctuation">.</span>objects<span class="token punctuation">.</span>countries<span class="token punctuation">)</span><span class="token punctuation">.</span>features<span class="token punctuation">;</span>
        <span class="token comment">// 3</span>
        <span class="token keyword">await</span> d3<span class="token punctuation">.</span><span class="token function">csv</span><span class="token punctuation">(</span><span class="token string">"https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/time_series_covid19_confirmed_global.csv"</span><span class="token punctuation">)</span>
          <span class="token punctuation">.</span><span class="token function">then</span><span class="token punctuation">(</span><span class="token parameter">csvData</span> <span class="token operator">=></span> <span class="token punctuation">{</span>
            <span class="token comment">// 4</span>
            countryDataset <span class="token operator">=</span> <span class="token punctuation">{</span><span class="token punctuation">}</span><span class="token punctuation">;</span>
            <span class="token keyword">let</span> latestDate <span class="token operator">=</span> csvData<span class="token punctuation">.</span>columns<span class="token punctuation">[</span>csvData<span class="token punctuation">.</span>columns<span class="token punctuation">.</span>length <span class="token operator">-</span> <span class="token number">1</span><span class="token punctuation">]</span><span class="token punctuation">;</span>
            csvData<span class="token punctuation">.</span><span class="token function">forEach</span><span class="token punctuation">(</span><span class="token parameter">province</span> <span class="token operator">=></span> <span class="token punctuation">{</span>
              <span class="token keyword">let</span> countryName <span class="token operator">=</span> province<span class="token punctuation">[</span><span class="token string">"Country/Region"</span><span class="token punctuation">]</span><span class="token punctuation">;</span>
              <span class="token keyword">let</span> caseCount <span class="token operator">=</span> province<span class="token punctuation">[</span>latestDate<span class="token punctuation">]</span><span class="token punctuation">;</span>
              <span class="token keyword">if</span> <span class="token punctuation">(</span>countryDataset<span class="token punctuation">[</span>countryName<span class="token punctuation">]</span> <span class="token operator">===</span> <span class="token keyword">undefined</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
                countryDataset<span class="token punctuation">[</span>countryName<span class="token punctuation">]</span> <span class="token operator">=</span> Number<span class="token punctuation">.</span><span class="token function">parseInt</span><span class="token punctuation">(</span>caseCount<span class="token punctuation">)</span><span class="token punctuation">;</span>
              <span class="token punctuation">}</span> <span class="token keyword">else</span> <span class="token punctuation">{</span>
                countryDataset<span class="token punctuation">[</span>countryName<span class="token punctuation">]</span> <span class="token operator">+=</span> Number<span class="token punctuation">.</span><span class="token function">parseInt</span><span class="token punctuation">(</span>caseCount<span class="token punctuation">)</span><span class="token punctuation">;</span>
              <span class="token punctuation">}</span>
            <span class="token punctuation">}</span><span class="token punctuation">)</span>
          <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
      <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span>

    <span class="token comment">// 5</span>
    <span class="token keyword">this</span><span class="token punctuation">.</span><span class="token function">setState</span><span class="token punctuation">(</span><span class="token punctuation">{</span>
      countryShapes<span class="token operator">:</span> countryShapes<span class="token punctuation">,</span>
      countryDataset<span class="token operator">:</span> countryDataset<span class="token punctuation">,</span>
    <span class="token punctuation">}</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
  <span class="token punctuation">}</span>

  <span class="token function">render</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
    <span class="token operator">...</span>
  <span class="token punctuation">}</span>
<span class="token punctuation">}</span>
</code></pre></div>
<ol>
<li>Stick an <code class="language-text">async</code> in front of <code class="language-text">componentDidMount()</code> to make sure you don't <code class="language-text">setState()</code> with <code class="language-text">null</code> values, i.e. wait to complete the promise.</li>
<li>Wait for the <code class="language-text">d3.json()</code> call to complete.</li>
<li>Wait for the <code class="language-text">d3.csv()</code> call to complete.</li>
<li>Reduce the dataset to map of (key: Country Name, value: Cases) pairs</li>
<li>Set the dataset to a new state variable, <code class="language-text">countryDataset</code></li>
</ol>
<p>You can try to use your fancy new <code class="language-text">countryDataset</code> as reference data for your choropleth map, so if you go ahead and give it a try...</p>
<p><strong>Map.jsx</strong></p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="diff"><pre class="language-diff"><code class="language-diff"><span class="token deleted-sign deleted"><span class="token prefix deleted">-</span><span class="token line"> const sampleData = {
</span><span class="token prefix deleted">-</span><span class="token line">   "Fiji": 3074,
</span><span class="token prefix deleted">-</span><span class="token line">   ...
</span><span class="token prefix deleted">-</span><span class="token line">   "Malawi": 5880,
</span><span class="token prefix deleted">-</span><span class="token line"> };
</span></span>
<span class="token deleted-sign deleted"><span class="token prefix deleted">-</span><span class="token line"> function getCountryFill(featureElement) {
</span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line"> function getCountryFill(referenceData, featureElement) {
</span></span><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line">   let name = featureElement.properties.name;
</span></span><span class="token deleted-sign deleted"><span class="token prefix deleted">-</span><span class="token line">   let value = sampleData[name];
</span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line">   let value = referenceData[name];
</span></span>
...

<span class="token deleted-sign deleted"><span class="token prefix deleted">-</span><span class="token line">  fill={getCountryFill(featureElement)}
</span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line">  fill={getCountryFill(this.state.countryDataset, featureElement)}
</span></span></code></pre></div>
<p>...you'll see it doesn't work out so well. Even the USA is missing data. If you went down the road above, go ahead and keep your changes. You'll build on them soon.</p>
<p>To fill in the gaps, you need to set the dataset's country names to those used in the TopoJSON. Add this workaround above the <code class="language-text">setState()</code> statement in <strong>Map.jsx</strong>'s <code class="language-text">componentDidMount()</code> function:</p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="diff"><pre class="language-diff"><code class="language-diff">async componentDidMount() {
<span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> var countryShapes = null;
</span><span class="token prefix unchanged"> </span><span class="token line"> var countryDataset = null;
</span><span class="token prefix unchanged"> </span><span class="token line"> await d3.json("https://cdn.jsdelivr.net/npm/world-atlas@2/countries-110m.json")
</span><span class="token prefix unchanged"> </span><span class="token line">   .then(async (json) => {
</span><span class="token prefix unchanged"> </span><span class="token line">     ...
</span><span class="token prefix unchanged"> </span><span class="token line">   });
</span></span>
<span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line"> // Left: the name of the country as indicated in the TopoJSON
</span><span class="token prefix inserted">+</span><span class="token line"> // Right: the name of the country as indicated in the JHU CSSE case data
</span><span class="token prefix inserted">+</span><span class="token line"> countryDataset["United States of America"] = countryDataset["US"];
</span><span class="token prefix inserted">+</span><span class="token line"> countryDataset["Greenland"] = countryDataset["Denmark"];
</span><span class="token prefix inserted">+</span><span class="token line"> countryDataset["Congo"] = countryDataset["Congo (Brazzaville)"];
</span><span class="token prefix inserted">+</span><span class="token line"> countryDataset["Dem. Rep. Congo"] = countryDataset["Congo (Kinshasa)"];
</span><span class="token prefix inserted">+</span><span class="token line"> countryDataset["Central African Rep."] = countryDataset["Central African Republic"];
</span><span class="token prefix inserted">+</span><span class="token line"> countryDataset["S. Sudan"] = countryDataset["South Sudan"];
</span><span class="token prefix inserted">+</span><span class="token line"> countryDataset["Côte d'Ivoire"] = countryDataset["Cote d'Ivoire"];
</span><span class="token prefix inserted">+</span><span class="token line"> countryDataset["Myanmar"] = countryDataset["Burma"];
</span><span class="token prefix inserted">+</span><span class="token line"> countryDataset["South Korea"] = countryDataset["Korea, South"];
</span></span>
<span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> this.setState({
</span><span class="token prefix unchanged"> </span><span class="token line">   ...
</span><span class="token prefix unchanged"> </span><span class="token line"> });
</span></span>}
</code></pre></div>
<p>Look at your map render. Some, but not all previously gray countries should now be some shade of pink; but oh no! Your outdated opacity function makes country case-quantities indistinguishable! Let's revise it.</p>
<h3 id="finaltouches">Final Touches</h3>
<p>Change the following in <strong>Map.jsx</strong>':</p>
<div class="kg-card kg-code-card gatsby-highlight" data-language="diff"><pre class="language-diff"><code class="language-diff"><span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> function getCountryFill(referenceData, featureElement) {
</span></span>	  let name = featureElement.properties.name;
	  let value = referenceData[name];
	  if (value) {
<span class="token deleted-sign deleted"><span class="token prefix deleted">-</span><span class="token line">	    let opacity = value / 10000;
</span></span><span class="token inserted-sign inserted"><span class="token prefix inserted">+</span><span class="token line">	    let opacity = Math.log(value) / Math.log(8) / 10;
</span></span>	    return `rgba(255, 0, 255, ${opacity})`;
	  }
	  return `rgba(200, 200, 200, 1)`;
<span class="token unchanged"><span class="token prefix unchanged"> </span><span class="token line"> }
</span></span></code></pre></div>
<p>Your map shows the most updated Johns Hopkins University confirmed cases.</p>
<h2 id="acknowledgements">Acknowledgements</h2>
<p>Thanks to Richard Zimerman for the <a href="https://medium.com/@zimrick/how-to-create-pure-react-svg-maps-with-topojson-and-d3-geo-e4a6b6848a98">basemap instructions</a> and the <a href="https://systems.jhu.edu/">Center For Systems Science and Engineering at JHU</a>.</p>
<!--kg-card-end: markdown-->]]></content:encoded></item><item><title><![CDATA[Spicing up Feature Film Credits with Classification — Part II]]></title><description><![CDATA[Why I’m bad with names, but it turned out for the best.]]></description><link>https://blog.danielkawalsky.com/spicing-up-feature-film-credits-with-classification-part-ii/</link><guid isPermaLink="false">Ghost__Post__5fbc81de3c4f984616f88c03</guid><category><![CDATA[Data Science]]></category><category><![CDATA[Namsor]]></category><category><![CDATA[NodeJS]]></category><dc:creator><![CDATA[Daniel Kawalsky]]></dc:creator><pubDate>Fri, 19 Apr 2019 03:49:00 GMT</pubDate><media:content url="https://blog.danielkawalsky.com/static/a6d9d7bbef2fc79df33048ddc087fbf2/1_Hh13-TBTQ9PVnApqdPPShg-1.png" medium="image"/><content:encoded><![CDATA[<h2 id="why-i-m-bad-with-names-but-it-turned-out-for-the-best-"><em>Why I’m bad with names, but it turned out for the best.</em></h2><figure class="kg-card kg-image-card kg-card-hascaption fluid-image" style="flex: 2.3688663282571913 1 0"><img src="http://localhost:2368/content/images/2020/11/1_Hh13-TBTQ9PVnApqdPPShg.png" alt="Spicing up Feature Film Credits with Classification — Part II" srcset="http://localhost:2368/content/images/size/w600/2020/11/1_Hh13-TBTQ9PVnApqdPPShg.png 600w, http://localhost:2368/content/images/size/w1000/2020/11/1_Hh13-TBTQ9PVnApqdPPShg.png 1000w, http://localhost:2368/content/images/2020/11/1_Hh13-TBTQ9PVnApqdPPShg.png 1400w" sizes="(min-width: 720px) 720px" maxWidth="1400"><figcaption>Nobody:The credits: Tom Skerritt, Sigourney Weaver, Veronica Cartwright, Harry Dean Stanton, John Hurt, Ian Holm, Yaphet Kotto, Bolaji Badejo, Helen Horton, Eddie Powell, Gordon Carroll, David Giler, Walter Hill, Ivor…</figcaption></img></figure><h2 id="background">Background</h2><img src="https://blog.danielkawalsky.com/static/a6d9d7bbef2fc79df33048ddc087fbf2/1_Hh13-TBTQ9PVnApqdPPShg-1.png" alt="Spicing up Feature Film Credits with Classification — Part II"/><p>The credit sequence is a sad excuse for an effort to connect people who “touch the film” with those who watch it in its final form. It’s such a dull format that it has become customary for theaters to brighten the room for moviegoers to comfortably <em><em>leave the theater</em></em>. I want to curb this custom and get people invested in the countless names that scroll by. In my <a href="https://towardsdatascience.com/the-trouble-with-classification-312cc6fe8b79" rel="noopener">previous article</a>, I explain why I ran with tagging names with nationality — put simply, strangers instantly connect when they find out they share a common origin story.</p><p>I want to make something like the sample credit sequence below. Let’s call this format “flag-tagged.”</p><figure class="kg-card kg-image-card kg-card-hascaption"><img src="/static/563c02ab8e5b578badc376a85d50a5d7/1_Utx00nYTz1geXUMDRrObiw.gif" class="kg-image" alt="Spicing up Feature Film Credits with Classification — Part II"><figcaption>The flag-tags are randomly assigned and do not accurately represent ethnonationality.</figcaption></img></figure><p>This article is about how one developer’s quick-and-dirty tech solutions can’t change a culture but instead act as a powerful tool to expose and correct naïvete. Whose? Mine, for starters.</p><figure class="kg-card kg-image-card kg-card-hascaption fluid-image" style="flex: 2.272727272727273 1 0"><img src="http://localhost:2368/content/images/2020/11/Spicing-up-Feature-Film-Credits-with-Classification---Part-II.png" alt="Spicing up Feature Film Credits with Classification — Part II" srcset="http://localhost:2368/content/images/size/w600/2020/11/Spicing-up-Feature-Film-Credits-with-Classification---Part-II.png 600w, http://localhost:2368/content/images/size/w1000/2020/11/Spicing-up-Feature-Film-Credits-with-Classification---Part-II.png 1000w, http://localhost:2368/content/images/2020/11/Spicing-up-Feature-Film-Credits-with-Classification---Part-II.png 1400w" sizes="(min-width: 720px) 720px" maxWidth="1400"><figcaption>Situation ‘A’ — I am the go-to credits-processing guy.Situation ‘B’ — Where I am now, developing a client-side post-production solution.</figcaption></img></figure><p>I will start off by addressing the hopelessness of my original project. Ideally, I would integrate nationality information into films themselves (flag-tagging) to make the credit sequence, hopefully, more enticing than the bathroom at the end of a movie. Since I don’t run an established editing operation and I’m not given footage as a matter of course for rapid credits-processing, I’m more inclined to write a script to automate flag-tagging by (1) programmatically find names in streamed video and (2) insert animated graphics to retroactively assign nationality information. This is above my paygrade.</p><figure class="kg-card kg-image-card kg-card-hascaption"><img src="/static/7ac8d5fb9073c4d344af7eba438f065f/1_4uHyagRVAI5gypdBdq-smQ.gif" class="kg-image" alt="Spicing up Feature Film Credits with Classification — Part II"><figcaption>Over time, the credits grow <a href="https://www.nytimes.com/2017/05/26/movies/why-end-credits-in-movies-are-so-long.html" class="ck kg" rel="noopener nofollow" style="box-sizing: inherit; color: inherit; text-decoration: underline; -webkit-tap-highlight-color: transparent;">longer</a> and list more distinct roles. [1]</figcaption></img></figure><p>But I trudged onward, dual-wielding classification tools and public databases in hopes of revealing an ethnonational narrative in the long history of feature films. The history of feature films is well over 100 years-old, involving roughly 300,000 credited names. It was a huge process of augmenting name data, and the challenges I came across led me to narrow my scope to a small, intriguing subset of people.</p><h2 id="please-onomastics-responsibly">Please Onomastics Responsibly</h2><p>From the name Mark Hamill, you cannot definitively gather ethnic nationality, but you <em><em>can</em></em> assign a probability that the name was given by parents of a certain ethnonational lineage. This kind of task is perfectly suited for a classification algorithm, which dissects inputs and uses machine learning models to sort them into categories that most closely aligns with their attributes. Let’s see how <a href="https://www.namsor.com/" rel="noopener nofollow">NamSor</a> predicts the heritage of the name of the famous American actor, <a href="http://www.filmreference.com/film/26/Mark-Hamill.html" rel="noopener nofollow">which we know</a> to be of English, Irish, Scottish, Welsh (paternal), and Swedish (maternal) descent.</p><figure class="kg-card kg-embed-card kg-card-hascaption"><iframe id="cp_embed_pBVGwd" src="https://codepen.io/danksky/embed/preview/pBVGwd?height=300&#x26;slug-hash=pBVGwd&#x26;default-tabs=js,result&#x26;host=https://codepen.io" title="Mark Hamill Tree" scrolling="no" frameborder="0" height="300" allowtransparency="true" class="cp_embed_iframe" style="width: 100%; overflow: hidden;"/><figcaption>Baby Name Wizard and Ancestry serve as the base-truth origin for the first and last names. For “Mark Hamill, U.S.A.,” NamSor’s most confident predictions are “British” and then “(German).”</figcaption></figure><p>See that it correctly guessed British? That’s 1 for 1! Only 299,999 to go!</p><figure class="kg-card kg-image-card kg-card-hascaption fluid-image" style="flex: 1.9553072625698324 1 0"><img src="http://localhost:2368/content/images/2020/11/1_qOO0ipAo0JSS5-PeRDiTlQ.png" alt="Spicing up Feature Film Credits with Classification — Part II" srcset="http://localhost:2368/content/images/size/w600/2020/11/1_qOO0ipAo0JSS5-PeRDiTlQ.png 600w, http://localhost:2368/content/images/size/w1000/2020/11/1_qOO0ipAo0JSS5-PeRDiTlQ.png 1000w, http://localhost:2368/content/images/2020/11/1_qOO0ipAo0JSS5-PeRDiTlQ.png 1400w" sizes="(min-width: 720px) 720px" maxWidth="1400"><figcaption>NamSor’s Diaspora endpoint drives the ethnonationality classification.</figcaption></img></figure><p>Yet, we can’t rely upon NamSor to be 100% accurate, even if it was a magical API. That’s because NamSor assumes tradition as a rule — that parents name their children to reflect their heritage. <strong><strong>NamSor has no idea that names it’s asked to classify might not be names given at birth.</strong></strong></p><h2 id="call-me-by-that-name">Call Me By That Name</h2><p>Historically, Jews, Italians, and Poles in Hollywood changed their names to more <a href="https://www.nytimes.com/2010/08/26/nyregion/26names.html" rel="noopener nofollow">pronounceable</a> stage names, sometimes <a href="https://en.wikipedia.org/wiki/Stage_name#Ethnicity" rel="noopener nofollow">distancing themselves</a> altogether from ethnic groups prone to discrimination by taking on a name with <a href="https://www.classic-rock-legends-start-here.com/aerosmith-3.html" rel="noopener nofollow">no trace</a> of their born heritage.</p><figure class="kg-card kg-embed-card kg-card-hascaption"><iframe id="cp_embed_NmMJqY" src="https://codepen.io/danksky/embed/preview/NmMJqY?height=300&#x26;slug-hash=NmMJqY&#x26;default-tabs=js,result&#x26;host=https://codepen.io" title="Natalie Portman Tree" scrolling="no" frameborder="0" height="300" allowtransparency="true" class="cp_embed_iframe" style="width: 100%; overflow: hidden;"/><figcaption>I don’t care what anyone says, Herschlag has a nice ring to it!</figcaption></figure><p>I couldn’t go on labeling people with presumed ethnonationality knowing that some large chunk of the names I had access to through <a href="https://www.themoviedb.org/" rel="noopener nofollow">The Movie Database</a> weren’t names that were given at birth. Lucky for me, <a href="http://imdb.com/" rel="noopener nofollow">IMBb</a> had the birth and stage name of every person I was looking for, so I just had to identify every credited person in the 5–7 most popular movies of every year since 1900 (n≈100,000) who changed their name for better marketability in the industry in order to correct the “name-changers.” Without knowing the motivation of any individual’s name-change, and to eliminate false positives, I assumed that they made the switch for reasons <strong><strong>other</strong></strong> than the following:</p><ol><li>They changed their name for marriage. (It’s actually <strong><strong>more</strong></strong> professional for showbiz women to <strong><strong>keep</strong></strong> their maiden name after marriage. <a href="https://www.instyle.com/weddings/14-celebrities-changed-their-name-after-marriage" rel="noopener nofollow">These women went against the grain.</a>)</li><li>They go by their initials or a nickname. (Consider writer-actor <a href="https://www.imdb.com/name/nm1145983/" rel="noopener nofollow">B.J. Novak</a>.)</li><li>They gained an honorific. (Consider <a href="https://www.cbsnews.com/news/patrick-stewart-knighted/" rel="noopener nofollow">Sir Patrick Stewart</a>.)</li></ol><p>What remained after that filter were ~7,000 significant name-changes, a substantial 7% of the sample and an interesting group to explore! I had some questions for the data before I dove in, including:</p><blockquote><em><em>Can we observe anglicization as a trend in name-changes?</em></em></blockquote><h2 id="no">No</h2><p><strong><strong>Hypothesis</strong></strong>: since a HUGE portion of the names in the population ended up British, you can observe a pattern of anglicization among name-changes in the American film industry.</p><p><strong><strong>Analysis:</strong></strong> Using NamSor to determine someone’s <em><em>actual</em></em> ethnonationality is dumb. Instead, I used NamSor to do what it’s built for; I got “impressions” of the ethnonationality associated with someone’s former and current <em><em>names </em></em>to look for patterns in ethnocultural importance to the industry.</p><figure class="kg-card kg-image-card kg-card-hascaption fluid-image" style="flex: 1.1636363636363636 1 0"><img src="http://localhost:2368/content/images/2020/11/1_tPrW9umhORziV_JT7R9rXw.png" alt="Spicing up Feature Film Credits with Classification — Part II" srcset="http://localhost:2368/content/images/size/w600/2020/11/1_tPrW9umhORziV_JT7R9rXw.png 600w, http://localhost:2368/content/images/size/w1000/2020/11/1_tPrW9umhORziV_JT7R9rXw.png 1000w, http://localhost:2368/content/images/2020/11/1_tPrW9umhORziV_JT7R9rXw.png 1280w" sizes="(min-width: 720px) 720px" maxWidth="1280"><figcaption>Too much?</figcaption></img></figure><p><strong><strong>Facts</strong></strong>: As it turns out, there is a net-outflux of British-sounding names to elsewhere-sounding names. My hypothesis is WRONG. Check it out.</p><figure class="kg-card kg-embed-card kg-card-hascaption"><iframe width="612" height="344" src="https://www.youtube.com/embed/YVqQWfpAs0I?feature=oembed" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""/><figcaption>We’re only looking at the eastern hemisphere because ethnonationality, even in melting pots like the USA, is categorized primarily by geographical ties developed over the Modern Era, <a href="https://worldpopulationhistory.org/map/1500/mercator/1/0/25/" rel="noopener nofollow">when populations were concentrated outside of the Americas</a>.</figcaption></figure><!--kg-card-begin: markdown--><p>About the above visualization:</p>
<ul>
<li>A red line means the name originates in Britain and terminates elsewhere.</li>
<li>A blue line means the name originates elsewhere and terminates in Britain.</li>
<li>A white line means there is no association with Britain.</li>
<li>The thickness of the line indicates the number of name-changes recorded in one year from one “ethnic nation” to another.</li>
</ul>
<!--kg-card-end: markdown--><p>There’s a lot to see here, but I want to point your attention to the red and blue lines, specifically between 1900–1970. Over this period, there appears to be a mass exodus of British-sounding to Portuguese-sounding names. Contrary to the hypothesis, this shows that there is a tremendous de-anglicization effect underway. Maybe because people aimed to distinguish themselves from the showbiz crowd with ethnic-sounding names?</p><h2 id="concluding-points">Concluding points</h2><p>After I saw my hypothesis was wrong, I felt that my project was at a dead-end. My goal was to deliver a product that could change the way people watch credits and I felt I had turned up with nothing. Even if I developed post-production software to change the visual content of the credit sequence, there are no guarantees that anyone would notice, let alone <em><em>care</em></em> that there is more to see after their movie has ended. Instead, I developed an independent website as a platform to present my progress as I <a href="https://social-good.github.io/website" rel="noopener nofollow">explored and understood my dataset</a>. This, of course, meant that anything I published was disassociated from the industry I was poking at. While <strong><strong>this</strong></strong> <strong><strong>made my impact negligible</strong></strong>, it forced me to find real value in the data and the tools I had on hand before I exposed something people <em><em>would </em></em>find interesting. The fun of exploring an untouched dataset quickly drained away when I found out my hypothesis was wrong, but I believe there might be more to extract from this group of name-changers than I originally thought.</p><p>Could there be a Part III? Only time will tell… In the meantime, thanks for reading! If the population I observed or my analysis process is of any interest to you, please get in touch!</p><p>[1] <em><em>The Movie Database (~5 most popular feature films of each year)</em></em></p><p>I took on this project with <a href="https://www.linkedin.com/in/jyotsna-pant/" rel="noopener nofollow">Jyotsna Pant</a>, a graduate student, for <a href="https://raghavan.usc.edu/2019-spring-computing-for-social-good/" rel="noopener nofollow">Computing for Social Good</a> at the University of Southern California.</p>]]></content:encoded></item><item><title><![CDATA[Spicing up Feature Film Credits with Classification — Part I]]></title><description><![CDATA[Why you’re better off leaving holes in “people data” unfilled and just leaving classification alone.]]></description><link>https://blog.danielkawalsky.com/spicing-up-feature-film-credits-with-classification-part-i/</link><guid isPermaLink="false">Ghost__Post__5fbc7f743c4f984616f88bcb</guid><category><![CDATA[Namsor]]></category><category><![CDATA[NodeJS]]></category><category><![CDATA[Data Science]]></category><dc:creator><![CDATA[Daniel Kawalsky]]></dc:creator><pubDate>Tue, 09 Apr 2019 03:38:00 GMT</pubDate><media:content url="https://blog.danielkawalsky.com/static/c3c865ccbfb9bb0c8f46c69c99e5520b/0_waSSRJUpLE8FpHLP-1.png" medium="image"/><content:encoded><![CDATA[<h2 id="why-you-re-better-off-leaving-holes-in-people-data-unfilled-and-just-leaving-classification-alone-"><em>Why you’re better off leaving holes in “people data” unfilled and just leaving classification alone.</em></h2><img src="https://blog.danielkawalsky.com/static/c3c865ccbfb9bb0c8f46c69c99e5520b/0_waSSRJUpLE8FpHLP-1.png" alt="Spicing up Feature Film Credits with Classification — Part I"/><p/><p>When you go to the movies, do you ever wish you could just push a button, skip the story, and watch the closing credits scroll by? If you answered “no,” you’re normal. Credits <em><em>are</em></em> boring.</p><p>But we take credits at the end of a movie for granted; after all, the credit sequence is factored into the running time of films. Giving credits a second thought, it seems bizarre that we cap a film, an artform that uses dramatic visuals and rhetoric to evoke a visceral response, with such a dull format as the credit sequence. Of course, the credits have earned their place in film as a traditional way of paying respect to its creators, sometimes even accompanied by thematic clips to make the last transition less stark. Everything before the credits is meant to be enjoyed passively, while everything after hardly resembles the art that precedes it — the traditional format suggests that you read a list of names and pay respect to each for the role they played in making a movie before it slides out of view. Only with time and repetition does the average moviegoer learn to associate names (especially those below the line) with style, ethos, and privilege.</p><p>Actors have the better end of the stick. Since their faces are projected in high detail while their voices project from surround sound speakers, you get familiar with the cast quickly; but, if you watched a movie for the second time, I bet you wouldn’t notice if all of the names of the crew roles were scrambled.</p><p>To try to close the “familiarity gap,” I thought that if I found the elements in names that help people recognize them, I could make credits more appealing for people. Personally, I remember people’s names better when I share experiences with them. Herein is the first complication: people will never meet most of the people whose names they see in front of them. The next apparent solution would be to bombard people with the public profiles built around these people, e.g. dynamically changing the videos of credit sequences and injecting IMDB biographies for each credited person. This seemed effective but too burdensome.</p><figure class="kg-card kg-image-card kg-card-hascaption"><img src="/static/588c7134456c385b78e8c18dc909ef39/0_iy_RaBkUq9X1sJ9T.gif" class="kg-image" alt="Spicing up Feature Film Credits with Classification — Part I"><figcaption>“that” = reading the life story of every writer, sound designer, and costume writer of Transformers 7</figcaption></img></figure><p>Next, I considered identity. If occupation and expertise aren’t interesting enough to keep people around for the credits, I figured a common background would be. Americans seem to light up when they meet someone who shares a common nationality, even if their ancestors immigrated 5 generations ago. I ran with this truism and decided to label every credited person I could find with their nationality. Then, I figured, I will have accounted for even the typical moviegoer who doesn’t care to look beyond the theater experience to “get to know” credited people.</p><figure class="kg-card kg-image-card kg-card-hascaption fluid-image" style="flex: 1.541501976284585 1 0"><img src="http://localhost:2368/content/images/2020/11/0_GzSo4wA1wDXBzC5p.png" alt="Spicing up Feature Film Credits with Classification — Part I" srcset="http://localhost:2368/content/images/size/w600/2020/11/0_GzSo4wA1wDXBzC5p.png 600w, http://localhost:2368/content/images/2020/11/0_GzSo4wA1wDXBzC5p.png 780w" sizes="(min-width: 720px) 720px" maxWidth="780"><figcaption>“Oh my god, <em class="gk" style="box-sizing: inherit; font-style: inherit;">I’m </em>Irish, too! “</figcaption></img></figure><p>I figured, since the “typical moviegoer” can’t know the ethnic background of each member of the credits, the labeling process should reflect the same naive perspective and make an educated geographical estimate based on the name itself. I figured wrong, but before I explain why, I’ll show you how far I took a hunch before it became glaringly obvious that my process was flawed.</p><p>I followed the intuition people use to assume the ethnicity and gender of someone they have never met and turned to machine learning models that associate names with places using survey, census, and poll data.</p><figure class="kg-card kg-image-card kg-card-hascaption"><img src="/static/7a6da907363f81421f35d9f5241130f9/0_PCbE4DxTtuLBSZRC.gif" class="kg-image" alt="Spicing up Feature Film Credits with Classification — Part I"><figcaption>“Assume,” “ethnicity,” and “gender” have been used in the same sentence. Exercise caution.</figcaption></img></figure><p>There are many tools to tack on identifiers to names, namely <a href="https://namsor.com/" rel="noopener nofollow">NamSor</a>, which draws on massive datasets to provide a best guess of someone’s gender and ethnicity based on solely their name. As sophisticated as NamSor has been engineered to work and as many <a href="https://en.wikipedia.org/wiki/Onomastics" rel="noopener nofollow">onomasticians</a>, linguists, and anthropologists as they consult, its (and any) AI classifier cannot operate with 100% accuracy. It’s impossible for two reasons:</p><ol><li>The name sequence is not long enough.<br>Take, for example, a similar problem of labeling a story with a genre. If the story went, “The person walked,” you’d be equally unconfident about placing it into any genre. But if the story went, “The girl forced trepid steps toward the dingey manor,” you’d presume this is the precursor of a horror story for a number of reasons, not the least of which is that manors <em><em>only </em></em>house evil.</br></li><li>Much more goes into a name than ethnicity, and much more goes into ethnicity than someone’s name.<br>Just as someone wouldn’t look exclusively to their name to determine how they identify themself, assigning identifiers from the outside-in isn’t a valid means of understanding a population.</br></li></ol><p>In the end, my plan to give more value to <em><em>individual</em></em> names than what appears on-screen caved under the responsibility to properly represent people. I think I had the right intentions, but my approach was all wrong. Rather than look for how people identified themselves, I thought, “this project calls for really good guessing,” and overlooked better solutions. Classification, as objective as it seems, is problematic when applied to qualitative human characteristics.</p><figure class="kg-card kg-image-card fluid-image" style="flex: 1.5267175572519085 1 0"><img src="http://localhost:2368/content/images/2020/11/1_rmcDdKnZCdYuwf7rlJmyKg.png" alt="Spicing up Feature Film Credits with Classification — Part I" srcset="http://localhost:2368/content/images/size/w600/2020/11/1_rmcDdKnZCdYuwf7rlJmyKg.png 600w, http://localhost:2368/content/images/2020/11/1_rmcDdKnZCdYuwf7rlJmyKg.png 1000w" sizes="(min-width: 720px) 720px" maxWidth="1000"/></figure><p>In my next article, I’ll show how NamSor performed by augmenting personal data with fine-grained attributes, like ethnicity, in terms of diaspora (where movie creators most likely came to the United States from). I’ll also show how this relates to base-truth information. Stay tuned!</p>]]></content:encoded></item></channel></rss>